The Hawthorne Effect

Your constantly-updated definition of the Hawthorne Effect and collection of videos and articles.
Be a conversation starter: Share this page and inspire others!

197 Shares

What is the Hawthorne Effect?

The Hawthorne effect is a bias where people temporarily change their behavior when they know they’re being observed. It often manifests as improved performance or exaggerated engagement but does not reflect normal behavior. UX (user experience) researchers and designers should understand and mitigate the effect regarding user behavior.

One way to validate your research data which hasn’t been affected by the Hawthorne effect is with triangulation. This is where you combine multiple methods, data sources, investigators, or theories to confirm and validate your findings. William Hudson, User Experience Strategist and Founder of Syntagm Ltd, explains more:

Transcript

How the Hawthorne Effect Became a Concern for Design

The Hawthorne effect became a psychological term—and consequently a design concern—after studies at the Western Electric Hawthorne Works in Cicero, Illinois, U.S.A. in the 1920s. Researchers altered working conditions in the telecommunications equipment factory, including by changing lighting and adjusting break times. They found that worker productivity consistently improved regardless of the change. For example, even in poorer lighting, the test group outperformed the control group—the workers whose conditions remained unchanged.

In the 1950s, sociologist Henry A. Landsberger, who coined the term “the Hawthorne effect,” reinterpreted the results. Landsberger believed that the test-group participants were more productive because they had received unusual extra attention as they participated in the study. The changes in the work environment hadn’t caused the workers’ improved performance; the workers’ awareness of being studied had made them behave more productively.

Although later research has questioned the extent and validity of the Hawthorne effect, documentation of this effect has emerged across various settings, from medical trials to education, and, crucially, in UX and user research. This means designers—who already have to be mindful of biases when they consider users, data, and potential solutions—need to watch out for a Hawthorne effect UX phenomenon—and mitigate it.

Solid research is vital for design solutions that make the best sense to users—and make the best sense of what users experience, need, and desire in their everyday lives. Watch as William Hudson explains the basics of user research, and how to approach it step by step:

Transcript

Designers Who Understand the Hawthorne Effect Do Better

The effect can seem advantageous for authority figures, from factory managers and business owners to teachers and parents. For example, imagine how a school classroom can become chaotic moments after a teacher briefly steps out—and quietens quickly when the students notice the teacher returning. Speed cameras are another example of how surveillance encourages good behavior.

However, the effect can sabotage design if researchers and designers fail to notice it, or if they let confirmation bias help it mask problems below the surface. For example, if designers have assumptions about what users want, they might look for things while observing users that confirm those assumptions—and ignore evidence that differs with what they believe. So, if users are already acting out of character during a user test, designers face a double disadvantage in that the “fake” behavior can lead them to look for and believe false things about the test prototype or solution. UX designers and researchers can run into the Hawthorne effect anywhere in the UX design process when they have contact with users—for example, in usability tests, field studies, or interviews. Anytime researchers observe users—and users know they’re under observation—the effect can arise and create problems for several reasons.

1. It Alters Authentic User Behavior

When test participants feel judged, even subtly, they may:

  • Try to impress the researcher or facilitator.

  • Avoid making mistakes, even if it means using workarounds.

  • Say what they think researchers want to hear.

These behaviors pollute research findings, especially if the researcher or designer relies on a small sample size or tests high-stakes flows like onboarding, e-commerce checkouts, or data entry. The users don’t act like natural users but like students in exam conditions. To avoid potential conflict—or judgment—they put on a show where they strive to do what they think is “right,” and not what they’d do if they were alone with the design solution or prototype.

An image showing two people standing beneath a wall full of CCTV cameras.

The sense of being watched can change everything, even how people perform the simplest actions.

© Photo by Burst, CC BY-SA 4.0

2. It Creates False Confidence in Design

For example, a user researcher might observe users breezing through a checkout flow during moderated tests and believe the website works well. However, if users are successful only because they feel pressure to perform, it’s a false positive and an unhelpful indicator. If the designer accepts the results as healthy signs of their website and the site goes live, the live product might see drop-offs, hesitation, or abandonment. The best designs must reflect great empathy for the user—and cater to how they truly think and feel.

Watch this video to understand more about how to design with empathy and advocate for users:

Transcript

3. It Skews Feedback and Testing Metrics

Hawthorne-driven behavior may:

  • Inflate success rates in usability-related tasks.

  • Suppress negative feedback in interviews.

  • Cause underreporting of confusion or frustration.

These distortions can mislead teams into thinking their designs are more effective than they really are. For instance, if users pretend not to care about design issues that annoy or confuse them, they become actors. The moment they act “out of character” is when data and feedback can distort the real picture. If designers follow the distorted information, they can end up creating tools that don’t accurately reflect reality. For example, designers often rely on personas—fictitious representations of real users—to guide design decisions.

Watch William Hudson explain why design without personas falls short:

Transcript

Where Do You Find the Hawthorne Effect in UX Research?

The effect can sneak into nearly every common UX research method:

Moderated Usability Tests

When researchers conduct usability tests which they or others moderate, users are acutely aware of observation and more likely won’t be “themselves.”

Contextual Inquiries

When researchers go to users in the latter’s “natural” environments, users know they’re part of a study and might adjust what they say or do to get a “passing grade.”

Think-Aloud Protocols

When researchers ask users to think out loud so they declare how they approach a problem or use a design, users may verbalize what they believe sounds smart rather than their genuine thoughts.

Eye-Tracking Studies

If users have equipment attached to them, such as eye-tracking gear to see where they look at a screen and for how long, they’ll have a heightened awareness of being watched. This may make them more likely to act like “device-wearers” than naturally behaving individuals.

Surveys and Interviews

Participants may answer questions to please the interviewer or align with perceived expectations. It could be because they want to be “nice” or “conventional,” or because they don’t want to explain an issue they might have—they might not want to bother bringing it up, or they might fear they’ll sound “stupid” if they do. Sometimes, and especially if the study concerns potentially negative or unhealthy behavior, they’ll distort facts to avoid potential judgment, such as: “I only drink a couple of beers on the weekend.”

Discover points to watch out for as Ann Blandford: Professor of Human-Computer Interaction at University College London explains pros and cons of user interviews.

Transcript

How to Mitigate the Hawthorne Effect in UX Research

While UX researchers and designers can’t eliminate the Hawthorne effect entirely, they can design smarter studies that reduce its influence.

1. Normalize the Observation

Over time, people typically become desensitized to having someone watch them. So, structure sessions that:

  • Include a warm-up task to ease users in.

  • Last long enough to let “performance mode” fade.

  • Repeat over multiple days or touchpoints, if possible.

This helps behavior become more natural as the novelty wears off.

Explore helpful techniques for how to approach users in this video with Ann Blandford:

Transcript

2. Use Remote and Asynchronous Tools

Unmoderated tools like Maze or UserTesting allow users to interact in their own environments. They’re less conscious of scrutiny, which yields more authentic interactions.

Better still, these platforms let you test at scale, which helps you more easily spot outliers and detect patterns that emerge only in real-world contexts.

3. Set Expectations Thoughtfully

The language you use during introductions sets the tone. Don’t word things in any way that might remind users of school exams, driving tests, or the like. Instead, try this script:

“We’re not testing you; we’re testing our design. If something’s unclear or frustrating, that’s a sign we need to improve it—not that you’re doing anything wrong.”

This reminder lowers pressure and increases honesty. It helps users drop their guard and act casually—secure in the knowledge that it’s all about the design.

4. Use Naturalistic Environments

Solid ethnographic research can bring precious insights, so observe users using the product:

  • In their own homes.

  • During actual tasks.

  • On their preferred devices.

Avoid artificial lab setups unless you have no alternative.

If you can’t observe users directly, cultural probes offer a great way to get authentic insights from them, as Alan Dix, Author of the bestselling book “Human-Computer Interaction” and Director of the Computational Foundry at Swansea University, explains:

Transcript

5. Anonymize Feedback Where Appropriate

Let users respond anonymously if you must gather sensitive insights. They’re more likely to express honest opinions about confusing UIs, broken features, or irritating workflows when they know their identities are shielded.

Discover important points about data collection in this video:

Transcript

6. Watch for Contradictions

If a user says something positive but behaves hesitantly, trust the behavior rather than their words. Their slower actions are likely to have come through the filter of social desirability.

7. Apply Methodological Rigor

Use caution when having contact with users. For example, well-planned longer sessions that put users at ease can help expose valuable insights, but remember to stay quietly observant and let them feel the absence of a “spotlight” on them.

8. Use Triangulation

Confirm and validate your findings with other methods, such as surveys and analytics to complement interviews. You have a wealth of ways to approach what you learn about users through qualitative research and quantitative research. UX researchers frequently use methodological triangulation like this to back up their findings.

Pick up valuable insights about the difference between qualitative and quantitative research, both essential ways to learn about users, as William Hudson explains:

Transcript

Case Example: Remote Testing Beats Moderated Labs

An SaaS company redesigned its dashboard based on lab testing insights. In moderated sessions, users easily found and used a “Create Project” button. However, in production, few users clicked it. Conversion stalled.

The team retraced their steps and ran unmoderated tests using a tool like PlaybookUX. Without an observer present, users struggled to locate the button—it was above the fold, far from where their mouse naturally moved.

The conclusion—the Hawthorne Effect had inflated performance during observed sessions. Users had “performed well” under the spotlight, not because the design was intuitive, but because they were extra focused.

This insight helped the team redesign the interface, increasing engagement by 42%.

Consider the Psychology behind the Hawthorne Effect

Help your research efforts—and what you design for your brand and users—and remember the complexity of the human mind when you approach users. The Hawthorne effect shares traits with:

  • Social desirability bias: The tendency to say or do what seems most acceptable.

  • Evaluation apprehension: Anxiety about being judged or evaluated.

  • Demand characteristics: Subtle cues that lead participants to guess the purpose of a study and act accordingly.

Together, these influences form a complex web of cognitive bias that researchers must navigate. For example, a researcher who approaches users as part of research towards a brand’s meditation or calm-inducing app might want to consider how answers like “I have fairly normal stress levels” might paint a different picture of the user’s true mental state—and skew the feedback they give. Perhaps they’re concerned about sounding neurotic or worried about divulging that they hate their job—all because they think an interview about stress management means questions about triggers. In any case, the researchers’ best defenses are awareness, humility, and triangulation.

Explore how awareness of users’ contexts helps to create designs that meet them on their level as Alan Dix explains:

Transcript

Ethical Considerations: Transparency Without Compromise

Twenty-first-century people already inhabit a heavily recorded world—a fact of life with not just CCTV cameras recording for security purposes but also numerous smartphones, doorbell cameras, and other devices capturing sights and sounds frequently throughout a typical day. Even if they’re not the center of attention in the recording, the fact that a device may be recording nearby can be enough to make camera consciousness become camera anxiety for some people. Few individuals like to be caught on film doing or saying the wrong thing. Consider this when you ask users about matters in recorded interviews.

Always ask users for consent to be recorded and be open and transparent about the recording’s use/s afterwards. In any case, researchers who consider a question like “Can I avoid disclosing what I’m testing to avoid bias?” should be quick to answer with a “no.” To mask study goals may help reduce demand characteristics, but full ethical transparency is non-negotiable.

Let users know:

  • They’re being observed and, if applicable, recorded.

  • Their feedback is valued, not judged.

  • Their data is safe and anonymized.

Ethical research builds long-term trust—both with users and stakeholders.

Find essential points about how to plan an observational study in this video:

Transcript

Above all, insight, humility, skill, and empathy can help researchers lessen the Hawthorne effect’s potential to derail design efforts. Humans are self-conscious social beings. In UX design, this insight is both a caution to tread carefully and an opportunity to access the truth of how users think, feel, and act.

Beware of the “watched = botched” dynamic; people won’t declare when they’re not being frank. You can’t know what they might be thinking deep down. “I can’t do this when you’re watching!” may be too awkward to come out as words, but it might in other ways, such as a user’s frustration after they’ve rushed an action—because they felt self-conscious and “under the microscope.” Maybe they’re afraid of feeling stupid if they can’t use your design or prototype—an unfair self-judgment that arises needlessly—and need you to reassure them.

Most importantly, stay humble. What users say—and even what they show—might not fully represent what they truly experience. Designers have a golden opportunity to bridge that gap, observe wisely, and design with empathy rooted in reality. UX design thrives on authenticity so designers can create experiences that reflect real-world needs—not staged performances. That’s not just good research—it’s great design.

Questions About Hawthorne Effect?
We've Got Answers!

What is the Hawthorne effect in simple terms?

The Hawthorne effect happens when people improve their behavior simply because they know someone is watching. In UX (user experience) design, this means users might act differently during usability tests than they would in real life. For example, a user might try harder to complete a task or avoid making mistakes if they know a researcher is observing them. This effect can lead to inaccurate data and poor design decisions.

To reduce the Hawthorne effect, UX designers should create natural, relaxed testing environments. Do not hover over participants or ask leading questions. Use remote or unmoderated testing whenever possible, as it helps capture authentic behavior. Analyze user behavior with caution and cross-check insights using multiple research methods.

Watch as William Hudson, User Experience Strategist and Founder of Syntagm Ltd, explains important points about how to back up user research with triangulation:

Transcript

Does the Hawthorne effect always improve performance?

No; the Hawthorne effect doesn't always improve performance. While observation often boosts motivation temporarily, it can also lead to anxiety, self-consciousness, or unnatural behavior—especially in UX design research. When users feel watched, they might rush tasks and botch actions they would do well if they were alone.

Whether users' performance improves or suffers due to their being watched, it can lead to overconfident design choices if designers fail to consider the Hawthorne effect. Designers may misinterpret observed success as a sign of an intuitive UI when users actually struggle under pressure. To avoid this, mix research methods—like remote testing, field studies, and behavioral analytics. These techniques reduce the observer's influence and reveal genuine user behavior.

For a wealth of insights and tips, enjoy our Master Class How to Conduct Effective User Interviews with Joshua Seiden, Co-Author of Lean UX and Founder of Seiden Consulting.

How do user behaviors change during design testing due to observation?

User behaviors often shift during design testing because people know they are being observed. This awareness can lead users to behave more carefully, follow perceived expectations, or hide confusion. In UX design, this distorts test results and creates a misleading picture of usability. For example, users might overthink clicks or hesitate to give honest feedback, and make interfaces seem more intuitive than they actually are.

Observation can also trigger performance pressure—users want to appear competent or helpful. As a result, they may push through tasks they'd usually abandon or avoid asking questions. This alters real-world behavior and undermines the accuracy of research.

To counter this, UX designers should use unobtrusive testing tools, minimize researcher presence, and encourage open-ended responses. Remote testing and anonymized feedback often reveal more authentic user interactions.

Find valuable insights in our article Unmoderated Remote Usability Testing (URUT) - Every Step You Take, We Won't Be Watching You.

Can the Hawthorne effect affect A/B testing in product design?

Yes; the Hawthorne Effect can influence A/B testing in product design. When users realize they're part of a test or sense that their behavior is monitored, they may interact with the two versions of the product differently. This distorts test outcomes and can lead product teams to favor a version that performs well under observation but fails in real-world use.

In UX design, this effect may cause users to explore features more thoroughly or stay longer on a page—not because the design improved, but because they feel watched. This leads to inaccurate conclusions about what actually drives engagement or conversion.

To reduce bias, conduct A/B tests without alerting users and allow behavior to unfold naturally. Combine A/B data with qualitative insights for a fuller picture of user intent and experience.

Learn how A/B testing can improve your designs, as William Hudson explains in this video:

Transcript

How should UX designers handle the Hawthorne effect in interviews?

To manage the Hawthorne effect in interviews, create a relaxed, non-judgmental environment. When users feel watched or evaluated, they may alter their answers to seem more helpful or competent. This distorts data and hides real frustrations or confusion with a product.

To reduce this bias, avoid leading questions, use neutral body language, and build rapport early in the interview. Ask indirect questions and encourage users to “think aloud”; it helps reveal true thoughts and behaviors. Also, when you conduct interviews in familiar settings or use video calls, you can reduce pressure on—and encourage honesty from—users.

If you record sessions for later review (but not do live note-taking), you can focus on the conversation while minimizing distractions. Always ask participants for their consent for recordings and remind them that there's no right or wrong answer.

Discover important considerations in our article Understand the “Why” of User Behavior to Design Better.

What research methods reduce the Hawthorne effect?

To reduce the Hawthorne effect, use research methods that observe users discreetly and minimize direct interaction. Remote usability testing, unmoderated studies, and A/B testing with concealed test variations let users behave naturally without feeling observed. Field studies and diary studies also offer authentic insights by embedding research into users' daily lives.

Behavioral analytics—like click tracking, session replays, and heatmaps—captures real actions without influencing them. These tools provide a clearer picture of how users interact with a product when they aren't aware of observation. Combine these methods with follow-up interviews to help validate findings while keeping observation bias low.

Use mixed methods—you can cross-check insights and identify any inconsistencies that observation might cause, and get more accurate, reliable UX design decisions to build upon.

Get a clear view of how analytics can help your design process in this video:

Copyright
Transcript

Should I tell users they're being observed during tests?

Yes; you must tell users they're being observed during tests—but how you frame it makes all the difference. Ethical UX research requires informed consent. However, overly formal or intense language can trigger the Hawthorne effect, and make users feel judged or perform unnaturally.

To reduce observation bias, keep your introduction casual. Emphasize that you'll test the product, not the person. Say something like, “We're looking to improve the design, so please explore naturally and speak your thoughts aloud. There are no right or wrong answers.”

In observations, don't sit too close or react strongly to what users do. Use screen-sharing or remote tools to observe without interfering. Also, remind users that their feedback helps improve the experience for others—this encourages openness.

Find vital points about how to plan an observational study from Alan Dix, Author of the bestselling book “Human-Computer Interaction” and Director of the Computational Foundry at Swansea University:

Transcript

Does longer-term observation reduce the Hawthorne effect?

Yes; longer-term observation often reduces the Hawthorne effect. When users become used to being observed, they stop changing their behavior to “perform.” This helps UX designers gather more natural, accurate data over time. In early sessions, users may act cautiously or try to “get it right.” However, as observation continues, they typically return to normal habits.

Longitudinal studies, field research, and diary studies work well because they extend beyond short, artificial test sessions. These methods blend into users' routines and so make them less aware of observation and more likely to behave authentically.

For product design, observe behavior across several sessions. Spread out research touchpoints and analyze patterns, not just single events. This reveals real usability issues that brief tests might miss.

Find a wealth of helpful tips in our Master Class How to Get Started with Usability Testing with Cory Lebson, Principal UX Researcher with 20+ years of experience and author of The UX Careers Handbook.

What are some recent or highly cited articles about the Hawthorne effect in UX design?

Macefield, R. (2007). Usability Studies and the Hawthorne Effect. Journal of Usability Studies, 2(3), 145–154.

This scholarly article examines the relevance of the Hawthorne effect in usability studies, questioning its generalizability and impact on research validity. It reviews the original Hawthorne studies and subsequent interpretations, analyzing their applicability to modern usability testing. The paper provides insights into how usability practitioners can design studies that minimize the influence of observation on participant behavior.

Adair, J. G. (1984). The Hawthorne effect: A reconsideration of the methodological artifact. Journal of Applied Psychology, 69(2), 334–345.

In this rigorous and influential article, John G. Adair re-evaluates the Hawthorne Effect, dissecting its methodological underpinnings and challenging its frequent overuse in social science research. He emphasizes that much of the evidence historically used to support the Hawthorne effect is anecdotal or methodologically weak. Adair's critical insights aim to clarify confusion between observation effects and related phenomena such as experimenter bias or demand characteristics. This work remains highly cited for urging greater caution and precision in interpreting participant behavior changes, especially in psychological and educational studies. It serves as a methodological cornerstone for experimental design discussions.

Gillespie, R. (1993). Manufacturing Knowledge: A History of The Hawthorne Experiments. Cambridge University Press.

Richard Gillespie's Manufacturing Knowledge offers a critical examination of the renowned Hawthorne experiments, pivotal studies in industrial and social psychology. Drawing from original records and researchers' personal papers, Gillespie reconstructs the experiments' evolution from tentative trials to authoritative publications. He challenges the notion of value-free science, illustrating how social, institutional, and ideological contexts shape scientific knowledge. This work is significant for its in-depth analysis of the interplay between research practices and the socio-political environment, providing valuable insights for historians, social scientists, and those interested in the sociology of knowledge and the development of industrial relations.

Johnson, J. (2010). Designing with the Mind in Mind: Simple Guide to Understanding User Interface Design Guidelines. Morgan Kaufmann.

Jeff Johnson's Designing with the Mind in Mind indeed merges insights from cognitive psychology with user interface (UI) design. The book explains how and why certain UI design choices work by referencing human cognitive capabilities and limitations. This alignment helps designers anticipate user behavior and craft interfaces that feel intuitive and reduce stress—important factors that can help minimize the Hawthorne effect during usability testing.

Earn a Gift Earn a Gift, Answer a Short Quiz!

1
2
3
4
1
2
3
4
Question 1
Question 2
Question 3
Get Your Gift
Interaction Design Foundation logo

Question 1

Why do workers often change their behavior during research studies or workplace observations?

1 point towards your gift

  • They want to help researchers prove their hypotheses.
  • They try to outperform others due to competition.
  • They know someone is observing them and respond to that attention.
Interaction Design Foundation logo

Question 2

How can the Hawthorne Effect influence usability test results?

1 point towards your gift

  • Test users might perform worse because they fear judgment.
  • Observed users might behave unnaturally and perform better than usual.
  • Users always behave the same regardless of observation.
Interaction Design Foundation logo

Question 3

What led researchers to first describe the Hawthorne Effect?

1 point towards your gift

  • A study showed workers always performed better in dim lighting.
  • Researchers noticed productivity increased every time they changed the work environment.
  • A survey found workers preferred team tasks over solo work.

Learn More About the Hawthorne Effect

Make learning as easy as watching Netflix: Learn more about Hawthorne Effect by taking the online IxDF Course Personas and User Research: Design Products and Services People Need and Want.

Why? Because design skills make you valuable. In any job. Any industry.

In This Course, You'll

  • Get excited to create products, services, and experiences people actually love, without the guesswork. Personas grounded in research will help you move beyond assumptions and craft experiences that truly delight. Expanding requirements cause 47% of projects to overspend, launch late, or fail altogether. One additional feature turns into five, and before you know it, the project is unrecognizable, packed with things no one asked for. When you design for a persona, you get to focus on what truly matters and solve the real problems people have. Fewer problems, more smiles. More smiles, more profit.

  • Make yourself invaluable when you can transform raw research into powerful personas that turn ideas into user-centered solutions that smash business goals and improve people's lives. As AI accelerates how fast we build and iterate, your timeless human-centered skills become even more powerful. You'll direct AI with deep human insight, and ensure outcomes remain meaningful, ethical, and genuinely resonate with people. This is how you stay in demand: Human-centered design skills transform AI from a tool into your new superpower. With design personas, you’ll make smarter decisions, keep everything and everyone on track, and drive your team and projects to success.

  • Gain confidence and credibility as you learn the simple step-by-step method to create effective personas, complete with templates that turn knowledge into real-world results. Get to grips with observations, triangulation, and grounded theory, distill your findings with affinity diagrams, and pour it all into personas that work. Access real user research in the optional course project, and bring your new skills to life with a portfolio piece that'll open up exciting and fulfilling career opportunities.

You'll Learn from the Best

In this course, you'll learn from one of the world's leading experts:

  • William Hudson: User Experience Strategist and Founder of Syntagm Ltd.

Get an Industry-Recognized IxDF Course Certificate

Increase your credibility, salary potential and job opportunities by showing credible evidence of your skills.

IxDF Course Certificates set the industry gold standard. Add them to your LinkedIn profile, resumé, and job applications.

Course Certificate Example

Be in distinguished company, alongside industry leaders who train their teams with the IxDF and trust IxDF Course Certificates.

Our clients: IBM, HP, Adobe, GE, Accenture, Allianz, Phillips, Deezer, Capgemin, Mcafee, SAP, Telenor, Cigna, British Parliament, State of New York

All Free IxDF Articles on the Hawthorne Effect

Read full article
4 Common Pitfalls in Usability Testing and How to Avoid Them to Get More Honest Feedback - Article hero image
Interaction Design Foundation logo

4 Common Pitfalls in Usability Testing and How to Avoid Them to Get More Honest Feedback

Usability testing is a cornerstone of the design thinking process. There are, however, common pitfalls that reduce the value of your test sessions. Here, we’ll go through four common issues you’ll encounter when you test your prototypes, as well as how to avoid each of them. At the end, we’ll also g

Social shares
717
Published
Read Article
Read full article
Light up Your User Research – Understanding the Hawthorne Effect - Article hero image
Interaction Design Foundation logo

Light up Your User Research – Understanding the Hawthorne Effect

If you’re going to observe your users in your UX research, and you probably should, you need to know that the act of observation can have an impact on individual behaviour. The Hawthorne effect is common in research and by understanding it, you can (to some extent at least) moderate its effect on yo

Social shares
678
Published
Read Article

4 Common Pitfalls in Usability Testing and How to Avoid Them to Get More Honest Feedback

4 Common Pitfalls in Usability Testing and How to Avoid Them to Get More Honest Feedback

Usability testing is a cornerstone of the design thinking process. There are, however, common pitfalls that reduce the value of your test sessions. Here, we’ll go through four common issues you’ll encounter when you test your prototypes, as well as how to avoid each of them. At the end, we’ll also give you a cheat sheet of things to say and do during your test sessions. Let’s get started!

Usability testing is not as simple as it might seem. That’s because people—for various reasons—might not give us the honest feedback that we need. For instance, they might not want to hurt your feelings. Or they might think you’re testing them rather than the design, and thus try to avoid “committing” errors.

Here are four things you should take note of, as well as how to solve them to make your test sessions more valuable.

1. People Don’t Want to Hurt Your Feelings

We all like to please others. That’s a great thing—it keeps society in order and makes us friendly even when we interact with strangers. However, in a test session, this basic human instinct might prevent your participants from giving you their honest (and harsh) feedback. After all, they know you must have spent a lot of time to create the design. What if their critique of a feature ends up hurting your feelings?

You might think this is a trivial pitfall, but you’d be mistaken! Time and time again, designers have found that participants give overwhelmingly positive feedback unless they’ve been correctly prepped before the test session.

Solution: Brief Your Participant for the Test Session

You should brief your participant so that they understand that they can—and should—provide their honest reactions and feedback.

One simple way is to deflect any perceptions about your personal involvement. You might de-emphasize how close you are to the work, or underscore your distance from it if you are from another team or company altogether. Some flexibility with the truth might be warranted, but be careful about barefaced lies denying how involved you are — seeking honesty from a test participant by being deliberately deceitful can backfire. The intention is to provide room and permission for them to be honest without feeling they need to approve or avoid negative comments that might hurt your feelings.

Another method is to tell your participant that you genuinely appreciate negative feedback because it allows you to improve the design. This way, you reframe negative feedback as learning opportunities and make it less likely that people will avoid mentioning anything that’s negative.

2. People Often Assume that You are Testing Them

Most test participants see a usability test and think “Oh, they’re testing to see whether I’m competent enough to complete the task!”. As designers, we might forget that most of our participants are not designers and thus don’t know what a usability test is about.

When a participant thinks you’re testing them (rather than the design), they tend to behave differently. They might be overly careful when they do tasks because they don’t want to appear stupid. This means that your results won’t be accurate, because the average user will likely be less alert when they use your product!

Participants also tend to give different feedback if they assume that you’re testing their abilities. For example, rather than say something like “This sentence is confusing and I’m not sure what to do next,” they might say, “I’m sorry that I blundered at this point; I wasn’t paying enough attention!” This makes it difficult to spot usability problems, as you’ll then need to read between the lines to find out what exactly went wrong.

Solution: Let Them Know You’re Testing the Design, Not Them

Start your test session by explicitly stating “We are not here to test you. We are here to test the design. We know this design might have some issues, and we need your help to identify those issues.”

This way, you clearly communicate that you’re not testing the participant. You’ll also help participants get into the mindset of testing the design. On top of that, when you let your participants know you’re asking for their help, you make it more likely for them to be forthcoming with their thoughts. They become exam invigilators rather than the students taking the exam.

Photo of a classroom with students writing, perhaps taking a test.

Make it clear that a usability test is not a test of the participant! Let participants know that you are testing your design.

© Dennis S Hurd, CC BY-NC-ND 2.0.

3. People Often Focus on Completion Rather Than the Journey

We naturally love to complete tasks. It fulfills us and gives us a sense of satisfaction. In the context of a usability test, this means participants tend to focus on completing their tasks instead of the route they took to get there.

One implication is that participants might adopt a “complete at all costs” mindset and carry out their tasks unnaturally. For example, a participant might be so focused on completing a task that they simply click on every link to check if it does the job. This means they’ll complete the task quickly, but since an average user is not likely to click on every link to see what it does, it means that your test results are not going to be accurate.

Another problem with this is that when people feel good after they’ve completed a task, they tend to provide nothing but positive feedback. Essentially, participants might forget all the bumps in the road—bumps which you want to smooth out—and tell you the journey was perfect because they’ve achieved their goal.

Solution: Focus on the Journey, Too

This sounds philosophical, but we’re actually talking about very practical matters here! You’ll need to put in some effort to counter our very human tendency to focus on end results.

One way to focus on the journey is to ask participants to think out loud. This means they should vocalize every single thought that comes to them while they complete your tasks. This is unnatural, and your participants might need to practice before they get good at it. Tell your participants something like this: “May I request that you think out loud? This means that you should vocalize everything that you’re thinking. For example, when you click a button, you can say something like ‘I want to click this button because I think that’s how I purchase this item.’”

Remind your participants to think out loud from time to time. If you realize they’ve stopped speaking, gently prod them with a “What are you thinking right now?” or “Can you please tell me what you’re thinking?”.

When you conduct a test, note down strange behaviors. For instance, if a participant clicks on a button and rapidly presses the “back” button, write it down. Later, at the end of the task, you can walk them back through the process to ask them why they behaved that way. Refer to your observations and bring up any issues that you saw them encounter on the way through.

4. People Can Show You More, If You Let Them

You’re the expert—and that’s why you’re conducting the tests. But this assumption of your own expertise can lead you to presume that you’ve picked up all the pain points through observation. This makes it less likely that you’ll ask your participants to show you more.

People have so much more to offer! If you end the usability test after the participant completes all their tasks, you are missing out on a huge opportunity to gather feedback and ideas.

Solution: End with a Post-Test Interview

At the end of each test, set aside some time to conduct a short interview. Ask your participant questions such as:

  • “On a scale of 1 to 5, how easy did you find it to complete the last task: 1 being most difficult and 5 being very easy?”

  • “Do you have any feedback or ideas on the product? They can be totally unrelated to our test session.”

Your participants are not designers, so it’s likely that their design idea won’t make it into the final product. However, that’s not really the point. When you ask people such questions, you help yourself understand what they think about the product, as well as areas that they’re concerned about. Sometimes, you might find their responses interesting or unexpected!

Photo of people raising their hands like in a classroom.

Ask your participants for additional feedback at the end of each of your test sessions.

© Derek Bridges, CC BY 2.0.

Putting It All Together: How to Get More Honest Feedback in Usability Tests

Below, we’ve arranged all our solutions so that you know what to do before, during and after a usability test.

Before a Session

  • Let participants feel comfortable giving negative feedback. Some things you can say include:

    • “Hi! I’m here today because I’m hired by [company] to test their design. Since I’m not related to [company] at all, you don’t have to worry about hurting my feelings! Feel free to give feedback, good or bad, so that we can learn and improve the design.”

    • “I’m here today to test a product that my colleague has created. Don’t worry, I won’t feel hurt if you say something negative about the design. In fact, I love hearing negative feedback because it lets me improve my colleague’s design!”

  • Let them know that you’re here to test the design and not them. You can say:

    • “Today, we’re not here to test you. We’re here to test the design. We know this design might have some issues, and we need your help to identify those issues.”

  • Politely request that they think out loud:

    • “May I request that you think out loud? This means that you should vocalize everything that you’re thinking. For example, when you click a button, you can say something like ‘I want to click this button because I think that’s how I purchase this item.’ Or, when you’re looking for something on a page, say something like ‘I’m trying to find the shoe category in this website.’”

During the Session

  • Remind the participant to think out loud if they fall silent. You can say:

    • “What are you thinking right now?”

    • “Can you explain what you’re doing right now?”

    • “Could you please think out loud, so I can follow what you’re doing?”

  • Note down any strange behaviors or questions that you’ve formed in your head while observing your participant.

    • Ask them either at the end of the task or at the end of the entire test session.

After the Session

  • If you’ve written down notes, ask them about these at the end of the test.

  • Ask them for more feedback:

    • “Do you have any feedback or ideas on the product? They can be totally unrelated to our test session.”

Download our template on how to get more honest feedback:

Advance Your Career With This Free Template for “How to Get More Honest Feedback in User Tests”
How to Get More Honest Feedback in User Tests
We respect your privacy
Get 1 powerful email each week: Design a life you love!

The Take Away

Usability tests can give you key insights that let you improve your design. However, it takes a conscious effort to extract better feedback out of participants during a test session. There are four common pitfalls you should take note of, and their solutions:

  • People don’t want to hurt your feelings.

    • Solution: Brief your participants before the test.

  • People often assume that you’re testing them.

    • Solution: Let them know you’re testing the design, not them.

  • People often focus on completion rather than the journey.

    • Solution: Focus on the journey, too.

  • People can show you more if you let them.

    • Solution: End with a post-test interview.

References and Where to Learn More

If you want to learn more about usability testing, check out our course on Conducting Usability Testing.

Feel Stuck?
Want Better Job Options?

AI is replacing jobs everywhere, yet design jobs are booming with a projected 45% job growth. With design skills, you can create products and services people love. More love means more impact and greater salary potential.

At IxDF, we help you from your first course to your next job, all in one place.

See How Design Skills Turn Into Job Options
Privacy Settings
By using this site, you accept our Cookie Policy and Terms of Use.
Customize
Accept all

Be the One Who Inspires

People remember who shares great ideas.

Share on:

Academic Credibility — On Autopilot

Don't waste time googling citation formats. Just copy, paste and look legit in seconds.