E-mail:
jsem@patrikpilous.cz
Goodhart’s law says that when a measure becomes a target, it ceases to be a good measure. Once people know they are judged by a number, they start improving the number instead of the thing it was meant to measure. The cobra effect is its close relative. A reward or rule meant to solve a problem ends up making it worse. In marketing you will meet both every time a metric grows while the business stands still.
Classification
- Type of phenomenon: a rule about measurement and motivation, driven by a cognitive shortcut
- Bias it reveals: we mistake a number for the reality it is meant to describe
- Mental area: Decision-making
- Marketing function: Measurement, KPIs, team management
- Buying journey: across all stages
- System 1 / System 2: arises in System 1, only System 2 can uncover it
In brief
- What it is: the number that determines rewards starts improving on its own, without any real result.
- Why we do it: people respond to what they are assessed on, not to what the person setting the target had in mind.
- Where you will meet it: numbers of leads, clicks, followers, articles or resolved tickets.
- How to avoid it: back up every metric with a second one that guards quality, and change targets regularly.
A real-life story
The cobra effect is named after a story from British India. The colonial administration in Delhi supposedly offered a reward for every dead cobra, people started breeding cobras so they had something to hand in, and when the authorities cancelled the reward, the breeders released the snakes. There were more cobras than before. In 2001 the economist Horst Siebert used the story as the title of his book Der Kobra-Effekt. However, nobody has found any contemporary evidence of cobra breeding, so it is more of an anecdote.
A similar case from Hanoi is documented. In 1902 the French colonial administration paid one cent for every rat killed, and a tail was enough as proof. Soon live rats without tails were running around the city, and rat farms appeared outside it.
The case was described in detail by the American historian Michael G. Vann of California State University, Sacramento. Instead of the cobra effect, he suggests talking about the rat effect, because that one really happened.
A modern version is the American bank Wells Fargo. For years it had a target of selling eight products to every household and fuelled it internally with the slogan “Going for gr-eight”. In 2016 the US consumer protection agency CFPB found that employees had opened over 2 million accounts and cards without customers’ knowledge, and fined the bank 100 million dollars.
A year later the bank raised its estimate to around 3.5 million potentially unauthorised accounts. The number of products per customer was supposed to show how strong the customer relationship was. Under pressure, employees delivered the number and damaged customer relationships for years.
How Goodhart’s law works
The British economist Charles Goodhart formulated the rule in 1975 for monetary policy. In its original wording, it says that any observed statistical regularity will collapse once pressure is placed on it for control purposes. In other words, a number that faithfully reflected the state of the economy stops working the moment the central bank starts deliberately moving it.
In 1997 the anthropologist Marilyn Strathern rephrased the rule into the sentence every manager knows today. It is the very sentence from the start of this entry: a measure that becomes a target ceases to measure well. She thus translated the original economic observation into the language of managing companies and schools.
The American social psychologist Donald T. Campbell independently described almost the same insight. His Campbell’s law from 1979 states that the more a quantitative social indicator is used for decision-making, the more it is subject to corruption pressures and the more it distorts the processes it was meant to monitor.
A measure works as long as it captures a side trace of real work. Once rewards depend on it, people look for the cheapest way to raise it, and that almost never runs through the real result. No ill will is needed. A rational response to badly set rules of the game is enough.
The cognitive shortcut sits with whoever sets the targets. A number is concrete and looks good in a meeting, so the brain easily mistakes it for what it is meant to describe. The leads chart is going up, so we are doing well. This substitution is the same mechanism described by the heuristic in which we quietly replace the hard question “is the business growing?” with the easier question “is the number growing?” and answer it with a clear conscience.
Goodhart’s law in marketing
In marketing almost everything is measurable, which is why Goodhart’s law is everywhere here. The table shows the measures I see most often, how they can be gamed and how to back them up.
| Measure as target | How it can be gamed | How to back it up |
|---|---|---|
| Number of leads (MQL) | collecting contacts who will never buy | share of leads that turn into deals |
| Click-through rate (CTR) | an enticing headline that does not match the content | time on page and conversions from clicks |
| Number of articles per month | short texts with no search volume | clicks from Google per article after 90 days |
| Search rankings | ranking for keywords nobody searches for | organic visits and enquiries |
| Number of followers | competitions, bought fans | reach and interactions per post |
| Campaign ROAS | targeting only people who would buy anyway | incrementality, comparison with a control group |
| Conversion rate | limiting traffic to the warmest audience | total revenue and margin |
I describe some useful metrics in my article on marketing KPIs. Even those are not immune to gaming. You can see this clearly with the conversion rate, which you can raise simply by bringing fewer people to your website.
In SEO, Goodhart’s law played out for years. When search engines gave a lot of weight to keyword repetition, websites stuffed keywords into their texts, and when they started counting links, link farms appeared. Search engines then changed the rules so that gaming stopped working, and this battle continues to this day, just with different metrics.
How to defend against Goodhart’s law
You cannot avoid it completely, because a company cannot be run without numbers. But you can make gaming much more expensive.
- Paired metrics. Add a quality measure to every quantity measure. Pair leads with the share of closed deals, and clicks with conversions.
- Reward results, not activity. A team paid for the number of campaigns will produce campaigns. A team paid for revenue and margin will think about which campaigns make sense.
- Track what is not rewarded too. Track some numbers with no link to bonuses. Nobody has a reason to push them, so they will show when the rewarded metrics start drifting away from reality.
- Rotate targets. A metric that has been a target for a long time gets exhausted. After a while, replace it or change how it is calculated.
- Ask what a cynic would do. Before introducing a target, try to imagine the laziest way to meet it. If one exists, someone will find it.
The cheapest of the five steps is the last one. Ten minutes is enough. Do it before you send the target to your team.
How it connects with other biases
Goodhart’s law is closely related to the McNamara fallacy, named after the US Secretary of Defense Robert McNamara. We track only what is easy to measure and treat what cannot be measured as if it did not exist. Customer satisfaction and brand strength are not in the spreadsheet, so they are the first to be cut, even though they are exactly what decides whether a customer comes back and recommends you to others.
Add to this the sunk cost fallacy. When a company builds its reporting and rewards around one metric, it finds it hard to abandon, even when it no longer measures what it was supposed to. Setting KPIs is therefore a decision at the level of marketing strategy, not just a dashboard setting.
Interesting questions on the topic
What is Goodhart’s law?
A rule stating that a measure ceases to measure well once it becomes a target. People start improving the number instead of the real result.
What is the cobra effect?
A situation where a measure or reward does not solve a problem but makes it worse. It is named after a story about a reward for dead cobras in colonial India.
How does Goodhart’s law relate to KPIs?
A KPI that rewards depend on tends to come under pressure to be gamed over time. That is why it is backed up with a paired quality metric and changed from time to time.
What is the difference between Goodhart’s law and Campbell’s law?
They describe the same phenomenon from different fields. Goodhart started from monetary policy and talks about a statistical regularity collapsing. Campbell started from evaluating social programmes and stresses that an indicator under pressure also distorts the very activity it was meant to monitor, for example teachers who start teaching to the test.
What is the McNamara fallacy?
The mistake of deciding only on the basis of what is easy to measure and ignoring everything else.
