You’ve completed your in-depth interviews, your contextual inquiry or your usability testing. What comes next? As UX practitioners know, when it comes to research, field work is only a fraction of the story.
How do you learn from mountains of data, and then make sure your insights create a tangible impact in shaping your product’s design?
We couldn’t think of anyone more qualified to ask than the prolific Steve Portigal, user researcher extraordinaire. From analysis and synthesis through to framing your findings, Steve walks us through a few post-research considerations to keep top of mind for your next research project.
What tips do you have for converting insights from research into action?
It’s a lot of work. According to Cooper’s Jenea Hayes, it’s roughly two hours of analysis and synthesis for every hour of research. I get grumpy when people talk about coming back from a research setting with insights. Insights are the product of analysis and synthesis of multiple sessions. It may just me being semantic-pedantic, but there’s something off-putting about the perfunctory way people describe: “Oh I come back from the session and I write up my insights and there you go.”
I see two different stages in making sense of research. Step one is to collate all the debrief notes, the hallway conversations, the shower thoughts you’ve had following the experience of doing the research. It’s a necessary first step and it’s heavily skewed by what sticks in your mind. It produces some initial thoughts that you can share to take the temperature of the group.
The next step is to go back to the data (videos, transcripts, artefacts, whatever you have) and look at it fresh. You’ll always see something different happened than what you think, and that’s where the deeper learning comes from. It’s a big investment of time, and maybe not every research question merits it. But if you don’t go back to the data (and a lot of teams won’t do it, citing time pressure), you are leaving a lot of good stuff on the cutting room floor.
I’m also a big fan of keeping the activity of sense making (what is going on with people?) separate from the activity of actions (what should we about it?). You want to avoid jumping to a solution for as long as possible in the process, so that your solutions reflect as deep an understanding of the problem as possible. Set up a “parking lot” where you can dump solutions as they’ll come up anyway. Depending on your research question, work your way to a key set of conclusions about people’s behaviour. Based on those conclusions, explore a range of possible solutions.
In your analysis, how do you decide what’s important?
Take time at the beginning of the research to frame the problem. Where did this research initiate? What hypotheses – often implicit ones – do stakeholders have? What business decisions will be made as a result of this research?
What research reveals doesn’t always fit into the structure that is handed to you ahead of time, so knowing what those expectations are can help you with both analysis and communication. Some things are important to understand because they’re part of the brief. But other things are going to emerge as important because as you spend time with your conclusions you realise “Oh this is the thing!”
I had a colleague who would ask, as we were getting near to the end of the process, but still wallowing in a big mess “Okay, if we had to present this right now, what would you say?” This is a great technique for helping you stop looking intently at the trees and step back to see the forest.
How do you make sure research data takes priority over stakeholders’ opinions?
So many aspects of the research process are better thought of as, well, a process. Talking to stakeholders about their questions – and their assumptions about the answers – is a great way to start. In that kickoff stage, explain the process. Share stories and anecdotes from the field. Invite them to participate in analysis and synthesis. Their time is limited, but there are many lightweight ways to give them a taste of the research process as it proceeds.
You don’t want the results to be a grand reveal, but rather an evolution, so that they can evolve their thinking along with it. If you’re challenging closely held beliefs (or “opinions”), make a case: “I know we expected to learn X, but in fact, we found something different.” Separate what you learned about people from what should be done about it so that you can respond to pushback appropriately.
What are some common mistakes you see that stops research staying front and centre during the design process?
To summarise a few of the points I’ve made above, some of the common mistakes I see are:
- Not including stakeholders in early problem-framing conversations
- Not including a broader team in fieldwork and analysis
- Delivering research framed as “how to change the product” rather than “what we learned about people” and “how to act on what we learned to impact the product”
- Researchers not having visibility into subsequent decisions
- Failing to deliver a range of types of research conclusions
How do you make sure your recommendations make it through to the next design iteration?
It’s challenging to ensure that research travels through any design or development process intact. Ideally, you’re involved as the work goes forward, sitting in meetings and design reviews to keep connecting it back to the output of the research, but think about the different aspects of the research that might take hold to help inform future decisions.
Is it stories about real people and their wants and needs? Is it a model or framework that helps structure a number of different types of users or behaviours? Is it a set of design principles? Or is it the specific recommendations? Often it’s a combination of several of these.
About Steve Portigal
Steve is the Principal at Portigal Consulting LLC – a consultancy that helps companies discover and act on new insights about their customers and themselves. He is the author of Interviewing Users: How to Uncover Compelling Insights and recently Doorbells, Danger, and Dead Batteries: User Research War Stories. In addition to being an in-demand presenter and workshop leader, he writes on the topics of culture, design, innovation and interviewing users, and hosts the Dollars to Donuts podcast. He’s an enthusiastic traveller and an avid photographer with a Museum of Foreign Groceries in his home.