When a metric becomes a target, dashboards stop measuring
The dashboard stops measuring what it used to measure. When a number starts to decide bonuses or reviews, the people measured tend to optimize the number, and the dashboard can stay green while the operation gets worse. It is not always dishonesty; often it is architecture: if the person who answers for the target also produces the data, the dashboard measures the report, not the operation.
What happens when a metric becomes a target?
The metric stops being a measure and becomes an aim. Optimization targets the number, not what the number stood for. This is what people call Goodhart's law. Manheim and Garrabrant describe it as the situation in which a metric is used to the point where further optimization is ineffective or harmful. The paper separates distinct failure mechanisms and does not deal with dashboards for small and midsize companies (Manheim and Garrabrant, arXiv, 2018, revised 2019).
Our reading, for anyone building a dashboard, is direct. Every metric is a stand-in for something you cannot see in full. As long as nobody is held to the stand-in, it tends to follow the original. Once somebody is, the two drift apart slowly, and the dashboard does not warn you.
Does this happen in real organizations?
It does, and independent bodies have recorded it. Bevan and Hood studied the English public health system, where central performance targets were in place. One example was the target of reaching 75% of category A ambulance calls within 8 minutes. An inspection cited in the paper found evidence that in a third of ambulance trusts, response times had been "corrected". The distribution of times showed a spike around 8 minutes, which the authors treat as a strong implication that responses taking between 8 and 9 minutes were reclassified to fall under the target (Bevan and Hood, Public Administration 84(3), 2006).
The second case is private. In September 2016, the US Consumer Financial Protection Bureau (CFPB) fined Wells Fargo $100 million over sales practices tied to the opening of accounts that may not have been authorized. According to the CFPB, the bank's own analysis pointed to about 1.5 million deposit accounts and about 565,000 credit card applications that may not have been authorized. The CFPB order goes back to January 1, 2011. The agency says the incentive compensation programs were monitored inadequately (CFPB, 2016).
Two caveats. These are large organizations and, in the English case, there were central performance targets. The cases illustrate the mechanism, not a rate you would find in your own company. And they do not show that every target distorts: they indicate that when the data depends on the people being measured, the distortion can take a long time to show.
Why does the dashboard not notice the distortion?
Because the dashboard reads the field someone filled in. If the status "resolved" is set by the person whose target is to resolve, the dashboard shows that person's decision, not the fact. The data arrives clean, on time, in the right format. What is missing is what makes it trustworthy: a source the person being evaluated does not control.
It is a neighbor of management dashboards: why the numbers don't match. There, the dashboard number diverges from another source. Here, it matches the source and has still lost its meaning. The error is not in the math. It is in who got to choose what goes into it.
What does this look like in an internal IT help desk?
A hypothetical scenario, to make the mechanism concrete. None of this is client data.
A midsize manufacturer measures IT support by "tickets closed within 24 hours". The target sits on the executive dashboard and weighs on the team's half-year review. In the ticketing system, the analyst can switch the status to "waiting on user", and the clock stops. The analyst can also close the ticket and ask the user to open a new one if the problem returns.
Nobody needs bad faith. The analyst who pauses the clock is following a rule of the system, and the one who reopens as a new ticket is avoiding a stain on the metric. Within three months, the dashboard shows rising compliance. In the hallways, users say they open the same request more than once before it is fixed. The dashboard is right about what it measures: fast closures. What the executives wanted to know was whether the problems actually end.
The same applies to anyone measuring innovation. Imagine a metric of "pilots completed this quarter" where the team itself marks the status. A pilot closed halfway to fit the deadline counts the same as one that led to a decision. That is also a hypothesis, and the rule is the same: the number comes to measure what is easy to mark.
What changes when measurement is designed?
Four decisions, all architectural, and they generally do not require changing tools. The first is to let the data be born from the event. Date, time and author are recorded by the system, without edits by the person being evaluated, and a status that stops the clock requires a reason and shows up in its own report. The second is to give the metric a counterweight. Closure within 24 hours travels with the seven-day reopening rate, and the dashboard shows both side by side, so that improving one while worsening the other stays in view.
The third is to write the definition down. What counts as "resolved", who can pause the deadline and for what reason becomes versioned text with an effective date, not team custom. The fourth is to check a sample. Every month, someone outside the area opens a small sample of records and compares what the dashboard said with what happened, and the result is logged.
There is a fifth decision, which is managerial and less technical: asking whether the metric needs to be tied to pay at all. That is our reading, not a conclusion of the studies cited. In both cases there was a lot at stake for the people measured (penalties or incentives), but they do not let us conclude that pressure grows in proportion.
In the Reatop case, waste collection and weighing are recorded in a field app that works without signal, and the admin dashboard generates the audit reports. The case illustrates the first of these decisions, data born at the source: operations record the fact in the field and the dashboard consumes it. It does not cover counterweights, versioned definitions or third-party sampling. For the wider picture of recording the decision a number triggered, see management dashboards: record the decision, and for the opposite problem, too many numbers, too many metrics stall decisions.
Frequently asked questions
My team resists change. How do I propose counterweights without sounding distrustful?
Present the counterweight as protection for people who meet the target for real. If a reopening metric exists, whoever resolves well looks better than whoever closes fast. Start with one metric, write the definition together with the team, and review it after one cycle.
I cannot justify the investment right now. Can I start without a project?
You can start with a small audit. Take a metric that weighs on reviews, open a small sample of records and compare them with what actually happened. If the gap is small, you have a first sign that the dashboard tracks the operation, at least for that metric and that sample. If it is large, you have the argument for the board, built on your own case instead of someone else's.
I do not see how to measure whether the dashboard is being distorted. Is it possible?
It is possible, through indirect signals. A spike of records just under the target threshold, reopenings that rise when the deadline shrinks, and clock-pausing statuses appearing more often than expected are clues. None proves distortion alone. Together, they indicate where the sample should look first.
The next step
If a metric that weighs on your team's evaluation comes from a field the team itself fills in, the Operational Architecture Diagnostic is a 30-minute conversation to map where that number comes from and who can change it.
Related case studies
Did you recognize your operation in this article?
Book the Operational Architecture Diagnostic: 30 minutes to map where your operation's bottleneck is. No proposal in the first conversation.
Book a diagnostic