Yesterday our Slack channel lit up with the combined knowledge of three members of our very own community.
Dr Jade Jenkins, Dr Marion Boberg and Stephanie Pratt were brave enough to work with me to trial a new idea that I had for our Ask the UXperts sessions.
The panel format proved to be a huge success. It allowed us to cover a broad range of topics and meant that we were able to get through more questions in the hour than is possible with a single guest.
The broad topic was UX research and we covered everything from selecting the most effective techniques to the emerging role of AI in user research. It’s fair to say that we all learned something!
If you didn’t make the session today because you didn’t know about it, make sure you join our community to get updates of upcoming sessions.
If you’re interested in seeing what we discussed, or you want to revisit your own questions, here is a full transcript of the chat.
Transcript
hawk
2017-07-05 18:02
Thanks everyone for joining us today. An especially big thanks to our panelists, @docboom @marionb and @stephaniempratt
hawk
2017-07-05 18:02
All of whom I cold emailed and lured into this session :wink:
flaxenink
2017-07-05 18:02
:+1:
hawk
2017-07-05 18:02
A quick intro of each:
Jade Jenkins is an experimental psychologist. After spending a decade researching various phenomena within social cognition (perception and memory) and occupational health psychology (including the interplay of technology, stress, and health), she earned her PhD in social and industrial-organizational psychology from Northern Illinois University.
Jade currently works in assessment within the Texas A&M University system, where her role consists of equal parts change agent, data and statistics octopus, and development consultant. She has expertise in numerous UX research methodologies, ranging from focus groups and experience sampling to biometric measurement (e.g., heart rate variability; HRV) and surveys.
She is a member of the Dallas-Fort Worth branch of UXPA, and welcomes kindred expressions of UX research methodology geekery on Twitter @jadejenkinsUX.
hawk
2017-07-05 18:03
Marion Boberg is a UX researcher born in Normandy France. She moved to Finland in 2003. Marion holds a PhD in Psychology from both French University of Caen and Finnish University of Eastern Finland.
Marion has 10+ years of experience in UX research, from Nokia Research Centre, Tampere University of Applied Sciences and Qvantel. Her UX experience is related mainly to IT products, mobile apps and telephony, her passion is about People and Innovation.
Author and Co-author of several HCI scientific publications and 10 + pending patents (5 granted) related to mobile UI and services for Nokia Technology.
hawk
2017-07-05 18:03
Stephanie M. Pratt is the Lead User Experience Researcher at a startup called LiveSafe. At LiveSafe, she primarily focuses on problem space research and understanding the users.
Prior to LiveSafe, she has worked in a variety of industries including another startup called mHelpDesk, GEICO, and a government contractor, Aptima. In those roles, she has done many levels of fidelities of usability testing and interviews. She has her Master’s degree in Human Factors / Applied Cognition from George Mason University.
Additionally, Stephanie is actively involved in the Washington, DC UX community.
hawk
2017-07-05 18:03
@docboom Can you give us some insight into the areas of research that you are particularly interested or passionate about at the moment?
docboom
2017-07-05 18:04
Absolutely, and thanks for having me here, @hawk!
docboom
2017-07-05 18:06
In a nutshell, you could say that I am a UX methodologist. I have many years of experience in quantitative, qualitative, and mixed-methods approaches to doing research. I am passionate about the UX research process itself, and am particularly passionate about new and emerging methodologies in UX research, especially heart rate variability; HRV.
docboom
2017-07-05 18:07
So in general, I’m here for anyone who has questions about UX research as a process. :slightly_smiling_face: I’m looking forward to the discussion!
hawk
2017-07-05 18:07
Perfect! @marionb – what are you areas of expertise?
marionb
2017-07-05 18:08
Hi, and thanks for inviting me :slightly_smiling_face:
marionb
2017-07-05 18:08
My expertise would be on designing for Playful Experiences (PLEX)
marionb
2017-07-05 18:09
or researching in that field, among many other subject –
hawk
2017-07-05 18:10
Fantastic. Sounds cool! And @stephaniempratt – what about you?
stephaniempratt
2017-07-05 18:10
Hi everyone! Excited to be here!
stephaniempratt
2017-07-05 18:11
My expertise falls under the qualitative side, with my psych background. From qualitative usability testing to interviews.
stephaniempratt
2017-07-05 18:11
I have conducted some “listening sessions” recently, which were really enjoyable, and led to building behavioral audience segments instead of personas, so happy to discuss that. As well as life as a researcher at a startup
stephaniempratt
2017-07-05 18:11
Looking forward to the discussion!
hawk
2017-07-05 18:12
Such a great range of skills. Thanks again for your time.
hawk
2017-07-05 18:12
ok <!here|@here> – who has a question to kick things off?
isha
2017-07-05 18:13
I’m curious to hear more from @stephaniempratt about behavioral audience groups vs personas. Seems like a lot of orgss are turnign away from personas. Can you speak to this a little bit.
steveportigal
2017-07-05 18:13
@marionb What is going on in organizations these days that is either helping or hindering the impact of user research?
hawk
2017-07-05 18:14
(if others have questions, ask away. I’ll queue them while Stephanie answers)
melanie
2017-07-05 18:14
What is your favorite framework for measuring user sentiment for a product feature?
stephaniempratt
2017-07-05 18:14
@isha – Sure thing! I mislabeled initially, behavioral audience segments is what I meant to call them. Pulled from Indy Young. :slightly_smiling_face:
bgas
2017-07-05 18:14
How would you suggest a small company with a limited budget can apply user research into the UX process when currently it is extremely limited?
melanie
2017-07-05 18:14
Do you see a role emerging for AI in user research?
stephaniempratt
2017-07-05 18:14
When dealing with various contexts that change, people will behave differently
stephaniempratt
2017-07-05 18:15
For example, I do work in the safety space, so in one instance a person may feel threatened, and in another context, may feel empathetic toward someone else in trouble
shillman
2017-07-05 18:15
How do you deal with businesses that want you to use techniques that you’re less convinced of the effectiveness? I’m thinking things like A/B testing (in most smaller companies) and focus groups.
flaxenink
2017-07-05 18:15
more talk about emerging methodologies in UX research, especially heart rate variability; HRV. @docboom Where do you see that going and how does that go with UX?
stephaniempratt
2017-07-05 18:16
So people don’t fall into a category all the time, they move, and it is outside gender/age/demographics, so by focusing on behaviors I think it helps build empathy and remove biases because anyone could experience that situation and behave that way.
stephaniempratt
2017-07-05 18:16
Sorry, editing my typos as I go :wink:
stephaniempratt
2017-07-05 18:16
Let me know if you would like more resources/explanation!
marionb
2017-07-05 18:17
@steveportigal From my personal point of view – the fact that business pace does not match research pace – often you need to downgrade the level of research or focus only on very specific issues at the time.
stephaniempratt
2017-07-05 18:18
@bgas asked “How would you suggest a small company with a limited budget can apply user research into the UX process when currently it is extremely limited?”
stephaniempratt
2017-07-05 18:19
In my experience it has been starting with little wins
stephaniempratt
2017-07-05 18:19
Maybe first taking some initial prototypes to a coffee shop and buying a few people a coffee for helping you out with feedback.
docboom
2017-07-05 18:21
@bgas I think it would be helpful to determine what your higher-level priorities are (both in importance and in terms of reach/impact on your org’s other goals) and then determine what conservative efforts could help you get the biggest bang for your buck.
stephaniempratt
2017-07-05 18:21
Or maybe an entry into a gift card raffle for filling out a survey
stephaniempratt
2017-07-05 18:21
Or even reaching out to current customers who are invested in the product
stephaniempratt
2017-07-05 18:21
a lot of time a thank you will suffice for them
stephaniempratt
2017-07-05 18:21
And just being listened to
leigh715
2017-07-05 18:21
What are some new methodologies you’re starting to see more of?
docboom
2017-07-05 18:22
yes! like @stephaniempratt suggested, you may be surprised by just how many survey respondents you can get with little money. especially if MTurk crowds interest you.
docboom
2017-07-05 18:23
@flaxenink Users are often unaware of the mental effort they are expending to complete a task because they are too focused on trying to perform well.
docboom
2017-07-05 18:23
HRV is a way to measure this process as users exert mental effort in real time (instead of having us researchers merely assume what’s happened after users have finished).
flaxenink
2017-07-05 18:23
@docboom: How can you track it? the HRV?
marionb
2017-07-05 18:24
@bgas try to involve small amount of user at the time from arly stage – the great opportunity of a small company is that you can also more esaly reach all the stakeholders and focus on what you all think is key to be tested.
docboom
2017-07-05 18:25
@flaxenink There are many physio ways to do it; I’ve often used a pulse wave sensor (usually an ear clip users can wear while completing a study). HRV is measured in milliseconds.
docboom
2017-07-05 18:25
High HRV = high effort, low HRV = low effort.
docboom
2017-07-05 18:25
HRV is of interest to human-computer interaction folks due to the assumption that well-designed products should be relatively easy to use (and, thus, users should demonstrate low HRV while using a product, all things considered).
docboom
2017-07-05 18:26
When coupled with other tools (e.g., video monitoring footage), HRV can help objectively identify weak points of the HCI.
stephaniempratt
2017-07-05 18:27
@shillman regarding your question ” How do you deal with businesses that want you to use techniques that you’re less convinced of the effectiveness? I’m thinking things like A/B testing (in most smaller companies) and focus groups.” I try to focus on what the stakeholders want to know
flaxenink
2017-07-05 18:27
@docboom: is the equipment expensive? While you are doing the testing?
stephaniempratt
2017-07-05 18:27
So for example, I will start building a research plan by having a meeting with them to get all their questions out there on a specific topic
stephaniempratt
2017-07-05 18:27
Then based on what they want to know, I will recommend/tell them the methodology I will pursue and explain to them why
stephaniempratt
2017-07-05 18:28
If they pushback on the methodology and prefer a different one, I will try to give them pros and cons of them and how they will or will not answer the questions
docboom
2017-07-05 18:28
@flaxenink Depending on the package, I think the equipment can either be on par with or much cheaper than eye tracking equipment and software.
stephaniempratt
2017-07-05 18:29
A good example might be if they ask “why are people abandoning the page?” and suggest a focus group, I will tell them why a usability test is more appropriate and provide more info on what a focus group could provide (e.g., concept feedback or marketing opinions)
shillman
2017-07-05 18:29
@stephaniempratt Aaah, this makes sense, yes.
stephaniempratt
2017-07-05 18:29
Sometimes it’s a game of politics though, and balancing that with what you want to research :wink:
shillman
2017-07-05 18:30
*nodnod*
flaxenink
2017-07-05 18:30
@docboom: Ah I see, will the HRV can be use in big companies or smaller ones do you think it will benefit from?
docboom
2017-07-05 18:32
@flaxenink Certainly the larger companies will likely have other equipment (e.g., video recording software or eye tracking) that are great compliments to HRV, thus enhancing its value.
docboom
2017-07-05 18:33
However, small companies struggling to pinpoint the exact moments where users are encountering design errors and other issues could benefit.
stephaniempratt
2017-07-05 18:34
@melanie: “Do you see a role emerging for AI in user research?” – What I have seen is a lot more research on AI experiences. At conferences I have attended there has been a lot of presentations on how to research using Amazon Echos/Google Homes, or chatbot prototypes. I have not seen AI as a support for user research as of yet, but maybe someone else has. :wink:
marionb
2017-07-05 18:34
@leigh715 about new methodologies, i see that there is more and more bridge, or collaborations between UX and CX, and I would say using user research to flesh out customer journey maps
flaxenink
2017-07-05 18:34
@docboom: if it was a smaller company how could they leverage the HRV? Do you have advise on that and what can they over come to get over the pinpoints of the exact moments?
shillman
2017-07-05 18:34
@melanie When you say AI, do you mean machine learning? Robotics? Something else?
melanie
2017-07-05 18:35
Yes, machine learning. For example, using data to improve your behavioral segments, something like that.
docboom
2017-07-05 18:35
@leigh715 Certainly eye tracking has exploded in recent years, though many still have misconceptions about what data is valuable, what it means, etc.
shillman
2017-07-05 18:36
@melanie I would not tend to expect the use of machine learning to do UX research, since it really just means using algorithms to deal with huge amounts of data. Usually to categorize it somehow or another.
hawk
2017-07-05 18:36
If you’ve just joined us and you have a research related question for our panel – jump in and ask at any time. I’ll queue them if nec.
stephaniempratt
2017-07-05 18:37
@melanie – you also asked, “What is your favorite framework for measuring user sentiment for a product feature?” – What are you trying to solve here? Are you trying to learn satisfaction of a new feature? Or something about a current feature? Please offer a little more context so we can better answer this question :slightly_smiling_face:
docboom
2017-07-05 18:37
@flaxenink I think I would start with a discussion of what the company’s biggest usability issues are, what’s been done in the past to address it, and how HRV fills that gap.
docboom
2017-07-05 18:38
So I’d think about what persistent, nagging issues you’ve had and, where possible, align that with a cost-benefit analysis
melanie
2017-07-05 18:38
Sure! Proposed features (not currently existing features). A way to measure the expected satisfaction or delight of the user.
ie, will the user like it? and using that to decide whether to build the feature
ie, will the user like it? and using that to decide whether to build the feature
docboom
2017-07-05 18:39
At minimum, HRV + video recordings is a good starting point and doesn’t have to be terribly expensive.
marionb
2017-07-05 18:39
@melanie I recently read this article on UX analytic – I see AI a way to guide us where to focus – but you still need to investigate the Why
marionb
2017-07-05 18:39
melanie
2017-07-05 18:39
@marionb thanks!!
stephaniempratt
2017-07-05 18:39
@leigh715 I’d also add on to take a look at Indi Young’s Practical Empathy, and learning about listening sessions. I have found them to be extremely enlightening to have a deep conversation with someone about how they think, feel, and react in context with your research question.
hawk
2017-07-05 18:42
<!here|@here> we have ~15 mins left in the session and we’re at the end of our queue of questions. Now is your chance to jump in with more!
stephaniempratt
2017-07-05 18:43
@melanie Ah, so you’re interested in understanding if they user will be interested in a proposed idea, so conceptual work. Well first, I’d make sure that the idea is based on solving a problem that has been identified from research. That will help back your reasoning for going forward with the feature.
stephaniempratt
2017-07-05 18:43
I actually read a great article on how to do design studios with constraints from user research this morning http://www.userfocus.co.uk/articles/user-research-design-studio.html
isha
2017-07-05 18:43
@stephaniempratt wanted to follow up on my first question. Can you speak a little more about how you document audience behavior segments (I might be butchering the term, sorry)?
melanie
2017-07-05 18:43
@stephaniempratt thanks!
flaxenink
2017-07-05 18:44
@docboom @marionb @stephaniempratt: What source or tip you can not live with out in the UX industry?
stephaniempratt
2017-07-05 18:45
Then from there, getting the concept in front of some people – building rapport with them so that they are comfortable being disapproving (examples: telling them you are not involved at all with the design, asking some warm up questions, etc.)
stephaniempratt
2017-07-05 18:46
Then getting the concept in front of them and maybe asking them what they think it does, what they think it is for, how useful or not useful they think it is (via likert scale), or anything else that relates to the questions/context at hand.
bgas
2017-07-05 18:46
When the product you are working on is designed for a very niche group of users, would UX research from people outside of the intended group (who do not understand the processes) still be beneficial?
stephaniempratt
2017-07-05 18:46
Kind of like a “sniff test” to see if it’s worth putting more effort to consider usability testing
stephaniempratt
2017-07-05 18:47
@isha Yes! Here is a wonderful resource from Indi Young : https://medium.com/@indiyoung/describing-personas-af992e3fc527
docboom
2017-07-05 18:48
@flaxenink Jeff Sauro and his team of UX researchers (MeasuringU) frequently produce excellent content on research as a process, so I often find myself tuning in to their latest posts.
marionb
2017-07-05 18:48
I’m a Fan of https://www.interaction-design.org/ and UX Mastery of course :wink:
marionb
2017-07-05 18:48
and I have some methodology books I fav
stephaniempratt
2017-07-05 18:49
I use both of those resources and these: http://www.userfocus.co.uk/articles/index.html , https://www.nngroup.com/articles/
isha
2017-07-05 18:50
when it comes to information design (dashboard, graphs etc.), do you have have any suggestions on resources (besides the ones posted above for other UX oriented information)?
docboom
2017-07-05 18:50
Ah yes, Nielson Norman was on the tip of my tongue! :slightly_smiling_face:
docboom
2017-07-05 18:50
*Nielsen
marionb
2017-07-05 18:50
Yes the new edition
marionb
2017-07-05 18:51
of Norman classic is also great!
docboom
2017-07-05 18:52
Oh! And for anyone who gets queasy around statistics, I have found that Andy Field’s books (or his website, Statistics Hell) are often received well by newcomers.
stephaniempratt
2017-07-05 18:53
@bgas “When the product you are working on is designed for a very niche group of users, would UX research from people outside of the intended group (who do not understand the processes) still be beneficial?” Getting it in front of other people may help identify big usability problems from human perception (e.g., contrast of colors, ability to navigate/find a button, etc.) but when it comes to getting through a process that is skilled, it is highly recommended to get niche users if you can.
viratahuja
2017-07-05 18:53
#ask-the-uxperts any advice any books
marionb
2017-07-05 18:53
@isha it’s a tool and online ressource: https://www.usertesting.com/ they tackle if i remember right some dashboards and other visual graphical data
stephaniempratt
2017-07-05 18:54
I used to work at a company that created software for field service professionals, so those were the people I would recruit would be service pros, but it was always hard to get the really low tech people to participate
stephaniempratt
2017-07-05 18:54
In the end, you have to make do with what you can get, so if getting it in front of a few people who aren’t that niche to get a gut check on higher level usability is all you can do, I’d say that’s better than no testing. :slightly_smiling_face:
flaxenink
2017-07-05 18:54
@docboom: Anymore tips about being queasy around statistics!
bgas
2017-07-05 18:55
@stephaniempratt Great response. Thank you.
stephaniempratt
2017-07-05 18:55
@flaxenink – the cartoon guide to statistics was good for me in grad school :slightly_smiling_face:
docboom
2017-07-05 18:55
@viratahuja If you’re only going to get one book on research as a process, I recommend Baxter, Courage, & Caine’s “Understanding Your Users”.
stephaniempratt
2017-07-05 18:56
You’re welcome!
viratahuja
2017-07-05 18:56
@docboom thanks
hawk
2017-07-05 18:57
We have just a few minutes left. Does anyone have anything pressing that hasn’t been answered?
hawk
2017-07-05 18:57
If you have follow up questions, all three panelists are active members of our community forums at http://community.uxmastery.com
marionb
2017-07-05 18:57
I would say for the single book : https://www.elsevier.com/books/observing-the-user-experience/goodman/978-0-12-384869-7
docboom
2017-07-05 18:58
@flaxenink Fail early and often. :slightly_smiling_face: Keep all old datasets for practice, and search for a stats expert/geek who doesn’t speak to you like a robot. :wink:
marionb
2017-07-05 18:59
I agree – I’d like to have a professional statitiscian just to guide me – but never found that (only at Uni)
marionb
2017-07-05 19:00
As a final word despite the budget or the schedule – keep going and do what you think is best – practice everyday and that’s how you’ll gain the expertise – and also interview as many people as you feel is necessary –
hawk
2017-07-05 19:00
And that’s a wrap! Thanks SO much to @docboom @marionb and @stephaniempratt for their time and advice today
hawk
2017-07-05 19:00
It was a fantastic session.
hawk
2017-07-05 19:00
I hope you all learned lots!
flaxenink
2017-07-05 19:01
Yes yes (clapping hands standing on a rainbow)!
