Participant screening is a process to filter participants for user research and market research. Researchers find the right participants with screener questionnaires.
User researchers use screener questions to get relevant data and make informed decisions about who should participate in a study. The selection of the right participants is essential to get accurate and useful insights. Participants should represent potential users and have relevant knowledge and user experience (UX).
In the video, Human-Computer Interaction and UX Expert William Hudson explains how best to screen research participants.
Why is Participant Screening Important in UX Research?
Participant screening is a foundational step in UX research. With thorough screening processes, researchers can guarantee their work effectively informs and improves the design process, which ultimately leads to better user experiences. Below are a few reasons why teams should invest in participant screening:
When researchers select participants who reflect the target user base, they can gather insights that are directly applicable to the users they design for—crucial for informed design decisions.
Screening focuses research on individuals whose experiences and feedback are most valuable. It’s important in studies with limited time and resources.
Participant screening contributes to the overall validity and reliability of the research findings. If the study includes a representative sample of the user population, researchers can be more confident their results are accurate and apply them to the broader user base.
Proper screening helps mitigate selection bias. This is critical to create inclusive and equitable designs.
In research that involves meaningful comparisons of different user groups—e.g., experts vs. novices—participant screening makes sure participants accurately represent these groups.
With a clear understanding of participants’ characteristics, researchers can tailor research methods and questions to elicit deeper insights. For example, understanding participants' familiarity with social media platforms can inform how to structure a user interview or usability test.
“You do research because you want to have impact. […] If you don’t have high-quality participants, that’s kind of a non-starter […] If you're not talking to the right people, it's really hard to make the right decisions.”
— John-Henry Forster, Former Senior Vice President of Product at User Interviews
How to Screen Participants
It can be a challenge for researchers to recruit participants for user research studies. They must find interested individuals, arrange the study appointments, incentivize prompt attendance and reliably organize it all.
Additionally, researchers must filter out attendees who lack the right type of experiences to provide valuable feedback or insights; subpar participants can impact research quality and design choices in adverse ways.
Researchers should follow the steps below to screen participants efficiently and effectively:
1. Choose Criteria for Participants
The first step to decide who to screen out. Behaviors, experiences and attitudes are the best criteria to define user groups. These are psychographics. Other information like gender, age or location are demographics.
User researchers often prefer psychographics. Good criteria to screen participants might include:
How familiar are they with a product: Screening participants based on their familiarity with the product helps researchers understand different levels of user experience, from novices to experts. This can shed light on how intuitive a product is for new users versus the additional functionalities that more seasoned users may leverage.
What they use a product for: Researchers need to understand what users use the product for to identify different use cases and the specific needs of each. This can help customize the product to meet varied user expectations and use conditions.
How often they use a product: This can help differentiate between occasional users and power users and provide insights into how the product fits into daily routines and its perceived value to different user segments.
Where they use a product (also called "context of use"): The knowledge of where users typically interact with the product—e.g., at home, at work, in transit—can uncover contextual factors that impact the user experience.
With the careful selection of criteria based on psychographics, researchers can create more effective screener questionnaires to identify the most relevant participants for their studies. This ensures realistic results grounded in the actual user behavior and attitudes, which leads to more informed and actionable design decisions.
2. Design Screener Questions
The objective of screener questions is to obtain genuine and unbiased insights from potential participants. Researchers should take into account that participants—unknowingly—often give answers they think a researcher wants to hear. That’s why, researchers should ask non-leading and open-ended questions.
For example, to test how often users use Facebook to search and find musical events, researchers would not want to ask:
“Is Facebook a good way to find musical events?”
But rather:
“How do you use social media?”
This is because they do not want to give away the purpose of the study upfront. The goal is for participants to answer honestly, and thus questions should not lead to a particular answer. Some participants might just want to participate in the study for an incentive and thus are willing to not answer honestly.
For example, the following screener question reveals the objective to the participants that the researchers might be looking for:
“Do you play mobile games at least once a week?”
Instead, a more neutral question is:
“What do you typically do in your free time on a weekly basis?”[LT3]
If asked the first question, participants would potentially have the idea the research is about mobile game apps and would likely respond in a way they believe aligns with the researcher's expectations or their interpretation of what seems like the “right” answer.
Open-ended and non-leading questions encourage participants to share their experiences and behaviors without bias. This type of question does not imply that there is a correct answer or a particular perspective that the researcher wants. As a result, participants are more likely to provide candid and varied responses, which can offer deeper and more meaningful insights into their habits, needs and preferences.
Overall, these questions allow user researchers to better select their participants, so they truly represent the user group and select those for further UX research methods like card sorting, user testing, etc. Ultimately, the goal is to select participants who can provide the most relevant and insightful feedback for the particular study.
3. Recruit the Right Participants
As the team completes their screener questions, they can start to think about participant recruitment. Researchers should use the screener questions to filter out participants who do not match the target audience necessary for testing new or existing designs.
The success of UX research relies on the recruitment of the right participants. Researchers can employ various methods based on the study's needs, timeline and budget. There are several options to recruit the right participants:
It’s imperative user researchers funnel out the wrong participants for the study, so the results are true and relevant to the design project.
© Interaction Design Foundation, CC BY-SA 4.0
Use a recruiting agency: Specialized agencies can streamline the recruitment process via their networks of potential participants. They can quickly identify and screen individuals based on the study’s requirements. While this method can save time and guarantee a high-quality pool of participants, it is often more expensive than other options.
Use an automated recruiting platform: Platforms automate the recruitment and screening processes using algorithms and databases. They offer access to a broad audience and can filter participants based on specific criteria. This method is efficient and cost-effective but may require additional screening to ensure participant quality.
Use existing users: Recruit from your product's current user base to provide valuable insights, especially for feedback on new features or usability testing. These participants already engage and are familiar with the product, which can lead to more relevant feedback. However, to maintain a balance and avoid bias, it’s best to include new users, too.
Use hallway recruiting or guerilla testing: Researchers approach people in public spaces or within their own organizations for quick and informal testing or interviews. It’s fast and low-cost but may not always provide participants who are representative of your target user base.
Use online forums and social media: Platforms such as Reddit, LinkedIn or specific forums can be excellent sources to find participants with particular characteristics or interests. This method is cost-effective and can reach a wide audience, but it requires more effort to screen and validate.
To achieve success, customize screener questions and the recruitment process according to your study's objectives and context. Continuously refine questions to align with research goals and participant profiles. Remain flexible and patient in recruitment to adapt to changing demographics and availability.
4. Review All Screener Responses
After collecting screener responses, researchers analyze the data to identify candidates who match the study criteria and bring diverse perspectives and experiences to the research. Researchers score responses based on relevance to the research objectives and create comprehensive participant profiles. Scoring allows for objective ranking, while profiling delves into nuances like psychographics and demographics.
Researchers compare profiles against study requirements to find candidates who meet the basic criteria and offer diverse insights. Iterative analysis may refine criteria or explore data deeper for a balanced participant group. This careful selection process guarantees the outcomes will be relevant, robust and actionable.
At this point in the process, researchers can also take the opportunity to choose “floaters” or backup participants even if they don’t fully match the criteria for the target audience.
For example, if the target audience of a study includes university students studying for their PhDs and one drops out of the study at the last minute, researchers could include a university student studying for a Master's degree. While they don’t match the criteria 100%, they might be a convenient replacement to fill the empty spot on short notice.
5. Communicate with Chosen Participants
Researchers may follow up with potential participants to verify their information and confirm their willingness and availability to participate in the study. Researchers should:
Schedule sessions and prepare participants: Finally, researchers schedule study sessions and provide participants with all necessary information, such as the study location (for in-person studies), technical requirements (for remote studies) and what to expect during the session—study duration, pre-study questionnaires, the setup of specific apps, etc.
Verify information: It's essential to confirm the accuracy of the information participants provided during screening. This can include the reconfirmation of demographic details, previous experiences or specific behaviors relevant to the study. This certifies participants fit the intended profile and will contribute meaningfully to the research outcomes or if the researchers should replace them.
Confirm willingness and availability: Researchers need to verify participants remain interested and available to participate. Review session dates and times, any compensation offered and confirm participants understand the commitment required. At this point, the team should openly discuss and sign any legal matters like non-disclosure agreements (NDAs) and privacy permissions.
Technical requirements: For remote studies, researchers must see to it participants have the necessary technical capabilities. Verify internet connection speed, guarantee compatibility with the virtual meeting software and provide troubleshooting support for technical issues.
When researchers meticulously follow these steps, they can facilitate a smooth and efficient study process with well-informed and prepared participants who are comfortable with their involvement in the research. Through goodwill, researchers can potentially keep the door open with participants for future research collaborations, as well.
Common Pitfalls in Participant Screening and How to Avoid Them
Common pitfalls in participant screening include:
The recruitment of participants who don't fit the profile needed for the study.
The underestimation of the importance of screening questions.
The disregard for the diversity required to obtain comprehensive insights.
Researchers can avoid these pitfalls if they clearly define the target audience and their characteristics before recruitment, create precise and relevant screening questions to filter out unsuitable candidates and supply a diverse participant pool for varied perspectives.
Actionable insights include the revision of screening questionnaires based on initial feedback with frequent consistency checks, the continuous observation of the recruitment process to ensure it attracts the right participants—to avoid “failing participants,” the alteration of the questions’ language to avoid “straightlined results” and report and score those who simply try to game the system.
Watch and listen as William Hudson explains the various pitfalls researchers encounter during the recruitment process and how best to avoid them.
For instance, if a study aims to understand the behaviors of experienced scuba divers, the screening process should first include open-ended and non-leading questions to confirm the participants scuba dive. Then, more specific questions about their scuba diving experience levels, such as the number of training hours they’ve logged, the depths and locations they’ve dove, the number of years they've scuba dove or the frequency of their dives.
If a participant responds they have the highest level of scuba diving experience along with the highest number of training hours, the deepest depths dove, the most locations dove, the most number of years and the highest dive frequency, then there’s a good indication the results might be straightlined.
How to Recruit for Accessibility Research
Designers need to design for all users and keep in mind the user experience looks, feels and sounds different to everyone, especially those with disabilities. It starts with recruitment, screening and serious user research. Accessibility is usability that focuses on people with disabilities.
© Interaction Design Foundation, CC BY-SA 4.0
Researchers recruit and screen participants for accessibility studies through specific strategies tailored to include individuals with disabilities. Examine the approach below:
1. Clearly identify what types of disabilities are relevant to the study. Accessibility includes a wide range of needs, including visual, auditory, motor and cognitive impairments. Researchers should test for exactly who they need—a hard-of-hearing person will most likely not have the same responses as someone with a learning disability, for example.
2. Use platforms and organizations that cater to individuals with disabilities. This could include disability advocacy groups or job fairs, online communities, schools and specialized agencies. The recruitment message should be accessible with clear language and alternative formats as needed.
3. Design accessible screening questionnaires for all potential participants. This includes the use of screen-reader-friendly formats, options for those with visual impairments and clear and straightforward language with language options.
4. Communicate with potential participants to accommodate their needs. For instance, you could provide information in Braille, use sign language interpreters or employ text-based communication for those who have auditory impairments.
5. Be flexible in scheduling sessions and consider the various needs of participants with disabilities. This includes transportation challenges and the need for breaks during sessions.
6. Establish the screening process respects participants' dignity and privacy. Be sensitive during the discussion of disabilities. Use gentle language and avoid making assumptions about participants' abilities.
In the video below, UX Expert and CEO of Experience Dynamics, Frank Spillers, discusses more about the recruitment process for accessibility tests in UX design.
Access the Accessibility Template mentioned in the video below:
