Transcript: Ask the UXperts: Taking Your Research to the Next Level — Panel Discussion

Thanks everyone for joining us today. An especially big thanks to our panelists, @docboom @marionb and @stephaniempratt

All of whom I cold emailed and lured into this session :wink:

:+1:

A quick intro of each:

Jade Jenkins is an experimental psychologist. After spending a decade researching various phenomena within social cognition (perception and memory) and occupational health psychology (including the interplay of technology, stress, and health), she earned her PhD in social and industrial-organizational psychology from Northern Illinois University.

Jade currently works in assessment within the Texas A&M University system, where her role consists of equal parts change agent, data and statistics octopus, and development consultant. She has expertise in numerous UX research methodologies, ranging from focus groups and experience sampling to biometric measurement (e.g., heart rate variability; HRV) and surveys.

She is a member of the Dallas-Fort Worth branch of UXPA, and welcomes kindred expressions of UX research methodology geekery on Twitter @jadejenkinsUX.

Marion Boberg is a UX researcher born in Normandy France. She moved to Finland in 2003. Marion holds a PhD in Psychology from both French University of Caen and Finnish University of Eastern Finland.

Marion has 10+ years of experience in UX research, from Nokia Research Centre, Tampere University of Applied Sciences and Qvantel. Her UX experience is related mainly to IT products, mobile apps and telephony, her passion is about People and Innovation.

Author and Co-author of several HCI scientific publications and 10 + pending patents (5 granted) related to mobile UI and services for Nokia Technology.

Stephanie M. Pratt is the Lead User Experience Researcher at a startup called LiveSafe. At LiveSafe, she primarily focuses on problem space research and understanding the users.

Prior to LiveSafe, she has worked in a variety of industries including another startup called mHelpDesk, GEICO, and a government contractor, Aptima. In those roles, she has done many levels of fidelities of usability testing and interviews. She has her Master’s degree in Human Factors / Applied Cognition from George Mason University.

Additionally, Stephanie is actively involved in the Washington, DC UX community.

@docboom Can you give us some insight into the areas of research that you are particularly interested or passionate about at the moment?

Absolutely, and thanks for having me here, @hawk!

In a nutshell, you could say that I am a UX methodologist. I have many years of experience in quantitative, qualitative, and mixed-methods approaches to doing research. I am passionate about the UX research process itself, and am particularly passionate about new and emerging methodologies in UX research, especially heart rate variability; HRV.

So in general, I’m here for anyone who has questions about UX research as a process. :slightly_smiling_face: I’m looking forward to the discussion!

Perfect! @marionb – what are you areas of expertise?

Hi, and thanks for inviting me :slightly_smiling_face:

My expertise would be on designing for Playful Experiences (PLEX)

or researching in that field, among many other subject –

Fantastic. Sounds cool! And @stephaniempratt – what about you?

Hi everyone! Excited to be here!

My expertise falls under the qualitative side, with my psych background. From qualitative usability testing to interviews.

I have conducted some “listening sessions” recently, which were really enjoyable, and led to building behavioral audience segments instead of personas, so happy to discuss that. As well as life as a researcher at a startup

Looking forward to the discussion!

Such a great range of skills. Thanks again for your time.

ok <!here|@here> – who has a question to kick things off?

I’m curious to hear more from @stephaniempratt about behavioral audience groups vs personas. Seems like a lot of orgss are turnign away from personas. Can you speak to this a little bit.

@marionb What is going on in organizations these days that is either helping or hindering the impact of user research?

(if others have questions, ask away. I’ll queue them while Stephanie answers)

What is your favorite framework for measuring user sentiment for a product feature?

@isha – Sure thing! I mislabeled initially, behavioral audience segments is what I meant to call them. Pulled from Indy Young. :slightly_smiling_face:

How would you suggest a small company with a limited budget can apply user research into the UX process when currently it is extremely limited?

Do you see a role emerging for AI in user research?

When dealing with various contexts that change, people will behave differently

For example, I do work in the safety space, so in one instance a person may feel threatened, and in another context, may feel empathetic toward someone else in trouble

How do you deal with businesses that want you to use techniques that you’re less convinced of the effectiveness? I’m thinking things like A/B testing (in most smaller companies) and focus groups.

more talk about emerging methodologies in UX research, especially heart rate variability; HRV. @docboom Where do you see that going and how does that go with UX?

So people don’t fall into a category all the time, they move, and it is outside gender/age/demographics, so by focusing on behaviors I think it helps build empathy and remove biases because anyone could experience that situation and behave that way.

Sorry, editing my typos as I go :wink:

Let me know if you would like more resources/explanation!

@steveportigal From my personal point of view – the fact that business pace does not match research pace – often you need to downgrade the level of research or focus only on very specific issues at the time.

@bgas asked “How would you suggest a small company with a limited budget can apply user research into the UX process when currently it is extremely limited?”

In my experience it has been starting with little wins

Maybe first taking some initial prototypes to a coffee shop and buying a few people a coffee for helping you out with feedback.

@bgas I think it would be helpful to determine what your higher-level priorities are (both in importance and in terms of reach/impact on your org’s other goals) and then determine what conservative efforts could help you get the biggest bang for your buck.

Or maybe an entry into a gift card raffle for filling out a survey

Or even reaching out to current customers who are invested in the product

a lot of time a thank you will suffice for them

And just being listened to

What are some new methodologies you’re starting to see more of?

yes! like @stephaniempratt suggested, you may be surprised by just how many survey respondents you can get with little money. especially if MTurk crowds interest you.

@flaxenink Users are often unaware of the mental effort they are expending to complete a task because they are too focused on trying to perform well.

HRV is a way to measure this process as users exert mental effort in real time (instead of having us researchers merely assume what’s happened after users have finished).

@docboom: How can you track it? the HRV?

@bgas try to involve small amount of user at the time from arly stage – the great opportunity of a small company is that you can also more esaly reach all the stakeholders and focus on what you all think is key to be tested.

@flaxenink There are many physio ways to do it; I’ve often used a pulse wave sensor (usually an ear clip users can wear while completing a study). HRV is measured in milliseconds.

High HRV = high effort, low HRV = low effort.

HRV is of interest to human-computer interaction folks due to the assumption that well-designed products should be relatively easy to use (and, thus, users should demonstrate low HRV while using a product, all things considered).

When coupled with other tools (e.g., video monitoring footage), HRV can help objectively identify weak points of the HCI.

@shillman regarding your question ” How do you deal with businesses that want you to use techniques that you’re less convinced of the effectiveness? I’m thinking things like A/B testing (in most smaller companies) and focus groups.” I try to focus on what the stakeholders want to know

@docboom: is the equipment expensive? While you are doing the testing?

So for example, I will start building a research plan by having a meeting with them to get all their questions out there on a specific topic

Then based on what they want to know, I will recommend/tell them the methodology I will pursue and explain to them why

If they pushback on the methodology and prefer a different one, I will try to give them pros and cons of them and how they will or will not answer the questions

@flaxenink Depending on the package, I think the equipment can either be on par with or much cheaper than eye tracking equipment and software.

A good example might be if they ask “why are people abandoning the page?” and suggest a focus group, I will tell them why a usability test is more appropriate and provide more info on what a focus group could provide (e.g., concept feedback or marketing opinions)

@stephaniempratt Aaah, this makes sense, yes.

Sometimes it’s a game of politics though, and balancing that with what you want to research :wink:

*nodnod*

@docboom: Ah I see, will the HRV can be use in big companies or smaller ones do you think it will benefit from?

@flaxenink Certainly the larger companies will likely have other equipment (e.g., video recording software or eye tracking) that are great compliments to HRV, thus enhancing its value.

However, small companies struggling to pinpoint the exact moments where users are encountering design errors and other issues could benefit.

@melanie: “Do you see a role emerging for AI in user research?” – What I have seen is a lot more research on AI experiences. At conferences I have attended there has been a lot of presentations on how to research using Amazon Echos/Google Homes, or chatbot prototypes. I have not seen AI as a support for user research as of yet, but maybe someone else has. :wink:

@leigh715 about new methodologies, i see that there is more and more bridge, or collaborations between UX and CX, and I would say using user research to flesh out customer journey maps

@docboom: if it was a smaller company how could they leverage the HRV? Do you have advise on that and what can they over come to get over the pinpoints of the exact moments?

@melanie When you say AI, do you mean machine learning? Robotics? Something else?

Yes, machine learning. For example, using data to improve your behavioral segments, something like that.

@leigh715 Certainly eye tracking has exploded in recent years, though many still have misconceptions about what data is valuable, what it means, etc.

@melanie I would not tend to expect the use of machine learning to do UX research, since it really just means using algorithms to deal with huge amounts of data. Usually to categorize it somehow or another.

If you’ve just joined us and you have a research related question for our panel – jump in and ask at any time. I’ll queue them if nec.

@melanie – you also asked, “What is your favorite framework for measuring user sentiment for a product feature?” – What are you trying to solve here? Are you trying to learn satisfaction of a new feature? Or something about a current feature? Please offer a little more context so we can better answer this question :slightly_smiling_face:

@flaxenink I think I would start with a discussion of what the company’s biggest usability issues are, what’s been done in the past to address it, and how HRV fills that gap.

So I’d think about what persistent, nagging issues you’ve had and, where possible, align that with a cost-benefit analysis

Sure! Proposed features (not currently existing features). A way to measure the expected satisfaction or delight of the user.
ie, will the user like it? and using that to decide whether to build the feature

At minimum, HRV + video recordings is a good starting point and doesn’t have to be terribly expensive.

@melanie I recently read this article on UX analytic – I see AI a way to guide us where to focus – but you still need to investigate the Why

@marionb thanks!!

@leigh715 I’d also add on to take a look at Indi Young’s Practical Empathy, and learning about listening sessions. I have found them to be extremely enlightening to have a deep conversation with someone about how they think, feel, and react in context with your research question.

<!here|@here> we have ~15 mins left in the session and we’re at the end of our queue of questions. Now is your chance to jump in with more!

@melanie Ah, so you’re interested in understanding if they user will be interested in a proposed idea, so conceptual work. Well first, I’d make sure that the idea is based on solving a problem that has been identified from research. That will help back your reasoning for going forward with the feature.

I actually read a great article on how to do design studios with constraints from user research this morning http://www.userfocus.co.uk/articles/user-research-design-studio.html

@stephaniempratt wanted to follow up on my first question. Can you speak a little more about how you document audience behavior segments (I might be butchering the term, sorry)?

@stephaniempratt thanks!

@docboom @marionb @stephaniempratt: What source or tip you can not live with out in the UX industry?

Then from there, getting the concept in front of some people – building rapport with them so that they are comfortable being disapproving (examples: telling them you are not involved at all with the design, asking some warm up questions, etc.)

Then getting the concept in front of them and maybe asking them what they think it does, what they think it is for, how useful or not useful they think it is (via likert scale), or anything else that relates to the questions/context at hand.

When the product you are working on is designed for a very niche group of users, would UX research from people outside of the intended group (who do not understand the processes) still be beneficial?

Kind of like a “sniff test” to see if it’s worth putting more effort to consider usability testing

@isha Yes! Here is a wonderful resource from Indi Young : https://medium.com/@indiyoung/describing-personas-af992e3fc527

@flaxenink Jeff Sauro and his team of UX researchers (MeasuringU) frequently produce excellent content on research as a process, so I often find myself tuning in to their latest posts.

I’m a Fan of https://www.interaction-design.org/ and UX Mastery of course :wink:

and I have some methodology books I fav

I use both of those resources and these: http://www.userfocus.co.uk/articles/index.html , https://www.nngroup.com/articles/

when it comes to information design (dashboard, graphs etc.), do you have have any suggestions on resources (besides the ones posted above for other UX oriented information)?

Ah yes, Nielson Norman was on the tip of my tongue! :slightly_smiling_face:

*Nielsen

Yes the new edition

of Norman classic is also great!

Oh! And for anyone who gets queasy around statistics, I have found that Andy Field’s books (or his website, Statistics Hell) are often received well by newcomers.

@bgas “When the product you are working on is designed for a very niche group of users, would UX research from people outside of the intended group (who do not understand the processes) still be beneficial?” Getting it in front of other people may help identify big usability problems from human perception (e.g., contrast of colors, ability to navigate/find a button, etc.) but when it comes to getting through a process that is skilled, it is highly recommended to get niche users if you can.

#ask-the-uxperts any advice any books

@isha it’s a tool and online ressource: https://www.usertesting.com/ they tackle if i remember right some dashboards and other visual graphical data

I used to work at a company that created software for field service professionals, so those were the people I would recruit would be service pros, but it was always hard to get the really low tech people to participate

In the end, you have to make do with what you can get, so if getting it in front of a few people who aren’t that niche to get a gut check on higher level usability is all you can do, I’d say that’s better than no testing. :slightly_smiling_face:

@docboom: Anymore tips about being queasy around statistics!

@stephaniempratt Great response. Thank you.

@flaxenink – the cartoon guide to statistics was good for me in grad school :slightly_smiling_face:

@viratahuja If you’re only going to get one book on research as a process, I recommend Baxter, Courage, & Caine’s “Understanding Your Users”.

You’re welcome!

@docboom thanks

We have just a few minutes left. Does anyone have anything pressing that hasn’t been answered?

If you have follow up questions, all three panelists are active members of our community forums at http://community.uxmastery.com

I would say for the single book : https://www.elsevier.com/books/observing-the-user-experience/goodman/978-0-12-384869-7

@flaxenink Fail early and often. :slightly_smiling_face: Keep all old datasets for practice, and search for a stats expert/geek who doesn’t speak to you like a robot. :wink:

I agree – I’d like to have a professional statitiscian just to guide me – but never found that (only at Uni)

As a final word despite the budget or the schedule – keep going and do what you think is best – practice everyday and that’s how you’ll gain the expertise – and also interview as many people as you feel is necessary –

And that’s a wrap! Thanks SO much to @docboom @marionb and @stephaniempratt for their time and advice today

It was a fantastic session.

I hope you all learned lots!

Yes yes (clapping hands standing on a rainbow)!