Goodhart's Law states that "when a measure becomes a target, it ceases to be a good measure." While I believe there is merit in that, does that mean we should abandon targets all together?
The concern Goodhart was trying to address was that when a metric is used as a goal or target, people may manipulate or game the metric in ways that undermine its usefulness.
I know that I have certainly seen this on a number of occasions.
While Goodhart's Law has been widely accepted as a valid observation, there are some arguments against it:
1 - Metrics can be designed to be resistant to manipulation:
While it is true that some metrics can be gamed, it is also possible to design metrics that are resistant to manipulation. For example, using a combination of metrics that are difficult to manipulate or incorporating feedback from multiple sources can help reduce the risk of gaming.
2 - Metrics can still be useful, even if they are gamed:
Even if a metric is gamed, it may still be useful as a tool for measuring progress or identifying trends. As long as the limitations and potential biases of the metric are understood, it can still provide valuable insights.
This one might have surprised you so let me give you a brief example:
A school district sets a goal to improve student test scores by 10% over the previous year. A teacher decides to focus only on teaching to the test, rather than providing a well-rounded education.
While the teacher's actions may result in higher test scores, it does not necessarily mean that students are receiving a better education overall. The test score metric is still useful for tracking progress, but it may not provide a complete picture of the quality of education being provided.
3 - Metrics can be used in a balanced way:
Rather than relying on a single metric as a target or goal, it is possible to use a balanced set of metrics that provide a more comprehensive view of performance. By using multiple metrics, it becomes more difficult to game any one metric and the overall view of performance is more accurate.
4 - The problem is with how metrics are used, not with the metrics themselves:
While Goodhart's Law suggests that metrics can be undermined when they are used as targets or goals, the problem is not with the metrics themselves, but with how they are used. By being aware of the limitations of metrics and using them in an appropriate way, it is possible to avoid the negative consequences of Goodhart's Law.
Overall, while Goodhart's Law is an important consideration when using metrics, it is not a reason to avoid using metrics and/or targets altogether.
Instead, it highlights the importance of using metrics in a thoughtful and balanced way, and being aware of their potential limitations and biases.
What do you think?
Are targets ok?
What’s your strategy for reducing metrics being gamed?