Hawthorne effect

The Hawthorne effect (Czech: hawthornský efekt) describes how people change their behaviour when they know someone is watching or measuring them. They work more carefully and follow rules they would otherwise get around. It is named after the Hawthorne Works factory near Chicago, where researchers studied in the 1920s how lighting affected the output of female workers. For a marketer, it is a quiet source of error in user tests, surveys, focus groups and sales team evaluations. Observational data then show behaviour under scrutiny, not everyday behaviour.

Classification

• Type of phenomenon: a behavioural phenomenon and research artefact that distorts measurement
• Bias it reveals: we mistake behaviour under scrutiny for everyday behaviour
• Mental domain: Social
• Marketing function: Customer research, team measurement
• Customer journey: across the journey, mainly research and customer care
• System 1 / System 2: System 1 (fast, automatic)

In brief

• What it is: people behave differently when they know they are being observed or are part of a study.
• Why it matters: your research and measurement results then describe the moment when someone is watching, and you plan the whole year based on them.
• Where you will meet it: user testing, surveys and focus groups, A/B tests of new features, team KPIs and mystery shopping.
• How to avoid it: supplement observation with anonymous data on everyday behaviour and compare against a control group.

A real-life story

Hawthorne Works stood in the town of Cicero on the outskirts of Chicago. It made telephones and switchboards for the Bell System, and according to Baker Library at Harvard, it employed over 40,000 people in 1929. Between 1924 and 1927, engineers in three departments changed the lighting and tracked how many relays and coils the workers could assemble. According to reports from the time, they found no clear link between lighting and output.

From 1927, the work continued in the relay assembly room, where five women worked separately from everyone else. The researchers adjusted their breaks and working hours. Output rose, but in a small group with its own room and the constant attention of both researchers and management, nobody could say which of the changes had actually worked.

The work of Elton Mayo of Harvard Business School then gave rise to the interpretation that attention and the feeling that people matter count for more than working conditions. Between 1928 and 1930, his team also conducted over 21,000 interviews with employees.

The effect only got its name later. It was popularised by the sociologist Henry Landsberger in his book Hawthorne Revisited, published by Cornell University in 1958. According to a review by Jim McCambridge and colleagues, the term first appeared in a methodology textbook as early as 1953, so its authorship is disputed.

How the Hawthorne effect works

The usual explanation goes in steps. A person knows someone is watching. They start guessing what the observer expects of them, and then they adapt to it because they want to look good. The review by McCambridge et al. in the Journal of Clinical Epidemiology describes this mechanism as a combination of conformity and the wish to make a socially desirable impression.

The clearest numbers come from a hospital. Stefan Hagel and colleagues compared direct observation with electronic disinfectant dispensers in a 24-bed intensive care unit. When a student with a form stood in the room, staff disinfected their hands 21 times an hour, but in the hours when only the wall dispensers recorded the counts and nobody was watching, it was just 8 times.

The study was published in 2015 in Infection Control & Hospital Epidemiology and compared 96 hours with an observer and 288 hours without one, during which the dispensers recorded 4,180 disinfections in total. According to the authors, the observer’s presence explained 61% of the differences in the number of disinfections between individual parts of a shift.

A related phenomenon is the placebo effect. In both cases, expectations change the outcome: with a placebo, expectations of the treatment; here, expectations of the observer.

The Hawthorne effect in marketing

Marketing lives on data about people’s behaviour, and much of it is collected in ways where people know they are being measured. The Hawthorne effect can creep in anywhere this happens.

1. User testing. A participant watched by a moderator goes through the checkout more patiently than they would at home on their phone. A session recording the visitor is unaware of at that moment, because they gave consent once in the cookie banner, often shows more hesitation and drop-offs. That is why, in a UX audit, I supplement moderated tests with recordings of real sessions and analytics.
2. Surveys and focus groups. Respondents answer the way they think they should. Saying “I would definitely buy it” costs little, while buying costs money, so hard purchase data is what counts.
3. A/B tests of new features. A new thing can attract attention mainly because it is new. A Microsoft team led by Soheil Sadeghi described the novelty effect as interest that fades over time, so a short test overestimates the benefit.
4. Team measurement and KPIs. When salespeople know their calls will be monitored from Monday, the numbers can jump. If they drop back after a few weeks and the company has meanwhile recorded the new system as a success in a presentation to management and set bonuses on that basis, it ends up paying bonuses for the observation itself.
5. Mystery shopping. When a sales assistant knows about the programme or recognises the fictitious customer, they serve them more carefully than anyone else. You then assess their performance at a moment when they are on their guard, and an ordinary Friday afternoon is missing from the report.

They share one problem. When the measurement itself changes the behaviour of the people you are measuring, and you do not know it or do not mention it in the report, you need a second data source that is not created under scrutiny. Otherwise you have nothing to compare the test result with.

What remains of the original study

According to the management textbook legend, the workers’ output rose with every change in lighting, even when the light was dimmed. Economists Steven Levitt and John List tracked down the original data on microfiche in a library in Milwaukee and in a forgotten archive in Boston. In 2011, they analysed it in the American Economic Journal: Applied Economics and showed that the original figures do not contain the pattern described in the textbooks. There is no jump in output after every change in lighting.

Still, there is something in them. In the first year, output in the tested departments rose by about 10%, while the factory as a whole grew by 1.4% a year. Later fluctuations, however, fit the season better, because in summer output fell in all rooms, including those where nothing was being studied any more. The season therefore explains part of the increase, and Levitt and List speak only of hints of a subtler effect.

A 2014 systematic review by McCambridge, Witton and Elbourne selected 19 studies designed specifically to measure the effect, 8 of them randomised. The results are mixed. Most found some effect, but the studies differed in method and in what they actually measured, and 18 of them came from healthcare, so they cannot be used to calculate the size of the effect for a company or an online shop. The authors conclude that taking part in research changes behaviour, but we do not know under what conditions or by how much.

How to guard against the Hawthorne effect

You cannot switch the effect off, because people often have to know about the measurement, whether because of consent to recording or because you cannot run a test or interview without their cooperation. The point is rather not to treat observational data as your only source of truth.

1. Supplement observation with data on everyday behaviour. Analytics and purchase data show what people do without a moderator, and the test adds the reasons. Stick to GDPR consent and work with anonymised or aggregated data. Do not track specific individuals.
2. Let tests run longer. Monitor an A/B test of a new feature for at least several weeks and check whether the difference is fading. Only make a decision based on the conversion rate once it has stabilised.
3. Have a control group. When you introduce new KPIs in one team, compare it with a team where nothing changed. Without that, you cannot tell whether the system worked or just the attention.
4. Ask about what people have already done. “When did you last buy?” gives a more honest answer than “Would you buy?”
5. Repeat the measurement once the effect has worn off. Repeat mystery shopping or service evaluations after a gap and on days when nobody expects them.

Then feed the research results into your marketing strategy with a note on the conditions under which they were obtained. Whoever reads the plan six months later will at least see what was measured under scrutiny.

How it relates to other biases

The Hawthorne effect has a lot in common with the Pygmalion effect. There, a boss’s or teacher’s expectations change performance; here, it is simply the feeling that someone is watching. They often go hand in hand, because the observer usually has some expectation and people sense it.

Group pressure, as shown by the Asch experiment, also plays a part. Under the eyes of others, people gave answers that contradicted what they saw. According to interviews after the experiment, some of them simply did not want to stand out, while others genuinely believed the group.

And critics today also read the Stanford prison experiment as participants playing the role they believed the researchers expected. Thibault Le Texier documented this in 2019 in American Psychologist, drawing on archives and interviews with 15 participants. It is the same trap McCambridge describes for the Hawthorne effect.

Interesting questions on the topic

What is the Hawthorne effect?

A change in the behaviour of people who know they are being observed or measured. Typically, they work more carefully or answer the way they think is expected of them.

Why is it called the Hawthorne effect?

It is named after Western Electric’s Hawthorne Works factory in Cicero, near Chicago, where worker productivity was studied from 1924 to 1933.

Does the Hawthorne effect really exist?

Taking part in research does change behaviour, as a 2014 review of 19 studies confirms. However, a 2011 re-examination of the original data did not confirm the famous story about lighting in the factory, and nobody has reliably measured the size of the effect.

How does the Hawthorne effect affect A/B tests?

A new feature attracts attention because it is new, and a short test shows a bigger benefit than the one that will last. That is why tests are left to run longer and you watch whether the difference fades.

What is the difference between the Hawthorne effect and the placebo effect?

With a placebo, the result changes because of expectations about the treatment. With the Hawthorne effect, it is the awareness that someone is watching you.