How to write participant screeners for better UX research results

One of the best ways to guarantee quality results from your user experience research is to recruit the right kind of people for your studies.

While most websites and software products should be easy enough for anyone to use, the best feedback comes from actual or representative users. Finding the right participants. however, can be a frustrating logistical challenge.

Creating effective participant screeners really helps smooth out the process.

What’s a screener and why do I need one?

A screener is just what it sounds like – a list of questions intended to identify your target users and weed out those who aren’t suitable for your study.

The first step in creating an effective screener is to take a step back and identify the types of people who will be using your product and who you want feedback from. If you already have personas, great! Pick your target persona and start there. If you don’t, here’s my invitation to go ahead and create them.

Depending on resources and the type of work you’re doing, you may choose to either:

• Use a screener in real time and talk directly to people, or
• Set up a remote survey tool with survey questions.

Either way, here are my best practice tips on how to write participant screeners and find the right people for your research.

Question focus

Even if you don’t have formal personas, take the time to identify the top few behaviours, contexts, motivations and attitudes of the type of people you are designing for and want feedback from. Focus your screener questions on those elements.

Let’s say, for instance, that you’re creating a travel-booking application. The way a parent books a family vacation is different from the way an assistant books last-minute business trips.

You may be interested in hearing from both types of users, but make sure you have enough of each type of user and know that context before you move forward. You may even want to ask them different questions or observe them in different contexts. Key questions here relate to who they book travel for and for what purpose.

Notice that I didn’t mention demographics. A person’s age range and gender won’t tell you how they use your travel app. In fact, the same person could act as different persona types in different contexts.

Take a 35-year-old mother. She could spend months researching perfect vacation hotels and screening for deals at home, but book the first direct flight she sees for a last-minute work meeting. Her priorities and behaviour could vary greatly depending on her context.

You may also consider whether someone is an existing or potential user, their technical skills, or if they have experience with a competitor. This behaviour may not be directly related to persona characteristics, but provides valuable additional insight for your research analysis.

Finally, add questions that eliminate any conflicts of interest, such as accidentally testing an employee of a specific company or ‘professional usability testers’ who attempt to join as many studies as possible for compensation.

Question order

Ask elimination questions at the very beginning to avoid wasting people’s time.

If you know that you’re targeting last-minute business travellers, put the question that identifies that user type at the very beginning. Then, order the questions from highest to lowest priority. Don’t worry about whether they ‘flow together’, unless there is a specific follow-up question to a previous answer. That way, you can be efficient and ask as few questions as possible to eliminate inappropriate candidates.

Keep in mind that the most effective screeners are the shortest. There is no ideal number of questions, as the granularity of necessary participants depends on the study goals. That being said, don’t use screeners to gather general information that you could easily ascertain during the research process. You’ll want to ask again to verify anyway, and longer screeners make it more likely that a potential participant will drop out.

Ask precise, easily-answered questions

The remaining best practices here are actually true for any survey questions, but I’m mentioning them because they’re especially important in screening participants.

Ask precise questions with clear answers. For instance, if you want to know how often someone uses an app, use numbers of hours instead of vague terms like, ‘sometimes’ or ‘frequently’.

Make the answers distinct so there’s no confusion. If you’re asking about age ranges, you could say that you are looking for 19 and under, 20-29, 30-39, and so on, rather than 0-20, 20-30, 30-40 etc. It seems simple, but the overlap can be very confusing and potentially lead you to incorrectly grouping people.

It’s also important to include ‘other’, ‘none of the above’, ‘I don’t know’ or ‘not applicable’ options. This way, candidates don’t feel compelled to pick the closest thing or something random and end up in your study when they should be disqualified or screened out.  

But don’t make it too easy

You never want to be leading or obvious about the kind of people you want to participate in your study.

By virtue of the fact that people are taking your screener, you know they want to take part in your study. And it’s human nature to try to conform to what you know someone is looking for. So don’t be too obvious about what you’re looking for in a participant.

Here’s an example. If you want to talk to someone who has purchased a phone in the last month, don’t ask them that! Instead, ask which of a long list of items they’ve purchased in the last month, with ‘phone’ as just one of many multiple choice options. That way, they won’t know which option you’re looking for.

You can also ask multiple-elimination and acceptance questions to obscure your desired answers. For instance, if you want to talk to people who spend more than 20 hours per week on something, the instinct is usually to create ranges with ’21 or more’ as the only answer that moves forward.

However, if you break up the options to something like: 0-5, 6-10, 11-15, 16-20, 21-25, 26-30, 31-35 and 36 or more, you have 4 answer options that get eliminated and 4 that move forward.

Multiple eliminations and acceptances make it harder to tell which is the ‘desired’ answer. I also avoid yes/no questions as much as possible, because it’s easier to obscure your true purpose with open-ended or multiple choice questions.

If possible, I like to add at least one open-ended question to a screener to see how a participant responds. Shy, quiet types are often the most uncomfortable in research settings and the hardest to glean insights from, so I like to look for someone who is willing to express their opinions, whether written or verbal.

Remember: this is your first impression

Finally, remember that a screener is often the first interaction a user has with the research process at your company or client, and you want to ensure it’s a positive one. 

Provide a warm, open introduction setting the context of the screener and the study. You don’t want people to assume that they’ll get to participate in a study just for being screened, but you do want them to feel comfortable answering questions and motivated to move through.

Provide context and clear expectations about next steps if they pass the screening: how does scheduling work, how long will they need to spend and where do the sessions take place, compensation, etc. You’ll never be able to answer every question upfront or please everyone, but you can ensure that the highest number of candidates actually show up to your sessions if you give them information upfront.

There are many moving parts to consider when putting together a UX study to ensure it’s successful. Writing a screener is one of the first, vital steps. It takes practice to make clear, efficient screeners, but it’s necessary so you can be sure your target users are represented in your studies.