Yesterday, by popular demand, we hosted a session in our Slack channel on the subject of usability testing. Our UXpert was Cindy McCracken and she did a fantastic job.
As well as answering questions and handing out valuable advice, Cindy compiled this handy list of resources:
Reading List
- Understanding Your Users by Kathy Baxter, Catherine Courage and Kelly Cain
- Observing the User Experience by Mike Kuniavsky
- Handbook of Usability Testing by Jeffrey Rubin and Dana Chisnell
- The Moderator’s Survival Guide by Donna Tedesco and Fiona Tranquada
- Measuring the User Experience by Tom Tullis and Bill Albert
- Beyond the Usability Lab by Bill Albert, Tom Tullis and Donna Tedesco
Tools for all kinds of testing
User Research Techniques
- http://uxmastery.com/resources/techniques/
- https://www.nngroup.com/articles/which-ux-research-methods/
Guidelines for prioritising study findings
If you didn’t make the session because you didn’t know about it, make sure you join our community to get updates of upcoming sessions. If you have follow up questions for Cindy, you can ask them over on our community forums.
If you’re interested in seeing what we discussed, or you want to revisit your own questions, here is a full transcript of the chat.
Transcript
hawk
2016-11-21 17:27
Session starts at 3 pm Wednesday 30 November PDT (or 10 am Thursday 1 December AEST)
You can use the command /tz help to get time zone conversion assistance here on @slackbot3
You can use the command /tz help to get time zone conversion assistance here on @slackbot3
hawk
2016-11-29 22:55
The beginner’s guide to usability testing: http://uxmastery.com/beginners-guide-to-usability-testing/
jacqui_dow5
2016-11-30 21:56
I love usability testing!
hawk
2016-11-30 22:57
Quick overview of how these things go down:
hawk
2016-11-30 22:58
– I’ll introduce @cindy.mccracken
– Cindy will give an overview of the topic and run through some definitions
– We’ll throw it open to you for questions
– Cindy will give an overview of the topic and run through some definitions
– We’ll throw it open to you for questions
hawk
2016-11-30 22:59
If things get crazy, I’ll queue your questions in a back channel and Cindy will answer them as she gets through them
jasmine
2016-11-30 23:00
Hey just heard that there’s a session happening today? How do I join?
jonny.bennett
2016-11-30 23:00
you’re in it @jasmine!
hawk
2016-11-30 23:00
We’re just about to kick off so your timing is perfect :slightly_smiling_face:
jasmine
2016-11-30 23:00
Oh! Haha, this is my first time. :smile:
lukcha
2016-11-30 23:01
Welcome @jasmine!
jasmine
2016-11-30 23:01
Thanks! Looking forward to it! :slightly_smiling_face:
hawk
2016-11-30 23:01
Ok, show time!
lukcha
2016-11-30 23:01
We’re all in for a treat. :slightly_smiling_face:
hawk
2016-11-30 23:01
First up, a huge thanks for @cindy.mccracken for your time today – we really appreciate it
hawk
2016-11-30 23:01
And thanks to the rest of you for joining us :slightly_smiling_face:
hawk
2016-11-30 23:02
Cindy recently published a great beginner’s guide on usability testing for us – you can find it pinned in this channel
hawk
2016-11-30 23:02
The formal intro: Cindy McCracken has worked in UX more than 10 years and is in her element when planning studies, conducting research, and analysing data.
hawk
2016-11-30 23:02
Currently consulting with User-View, Inc., focused on UX in the medical and financial fields, she has worked as a senior user researcher at Fidelity Investments, BB&T and iContact. Cindy earned a master’s degree in information science from the University of North Carolina-Chapel Hill.
hawk
2016-11-30 23:03
When she’s not working, you can find her hanging out with her 9-year-old daughter, reading historical fiction, or winding down in a yoga class.
hawk
2016-11-30 23:03
We asked her to come today to talk usability testing because it’s something that you (our community) requested.
hawk
2016-11-30 23:03
So @cindy.mccracken – over to you for an intro to the topic
cindy.mccracken
2016-11-30 23:03
Thanks for the intro, @hawk!
cindy.mccracken
2016-11-30 23:04
Hi everyone! I’m excited to have a conversation today about the details of usability testing.
cindy.mccracken
2016-11-30 23:04
I did my first usability tests for a non-profit when I took over for their webmaster in 2003.
cindy.mccracken
2016-11-30 23:04
I read the book “Designing Web Usability” by Jakob Nielsen, conducted tests, and was hooked.
cindy.mccracken
2016-11-30 23:05
So usability testing became a focus of my career.
cindy.mccracken
2016-11-30 23:05
Usability testing is a very useful technique for learning how well your designs work for people, but the more you get into it, you realize there’s always room for improvement.
cindy.mccracken
2016-11-30 23:05
You can always get better results.
cindy.mccracken
2016-11-30 23:05
There’s a lot to getting the right participants, writing effective tasks, and observing while remaining neutral.
cindy.mccracken
2016-11-30 23:06
The majority of usability testing is called formative testing – where you’re mainly concerned with learning how your product could be improved.
cindy.mccracken
2016-11-30 23:06
You come up with goals to test, and write realistic tasks for participants to get answers to your questions.
cindy.mccracken
2016-11-30 23:07
Of course, there are a lot of variations on the traditional in-person study.
cindy.mccracken
2016-11-30 23:07
For example, remote moderated testing. That gives you a lot more options for recruiting.
cindy.mccracken
2016-11-30 23:07
And then there’s the testing of mobile apps, which is very important these days.
cindy.mccracken
2016-11-30 23:08
And remote unmoderated testing – where you set up tasks in an online tool like UserZoom for hundreds of participants.
cindy.mccracken
2016-11-30 23:08
But for all types of usability testing, you need five things:
cindy.mccracken
2016-11-30 23:08
a design to test, participants, a test plan, a moderator (or test tool), and findings
cindy.mccracken
2016-11-30 23:09
For the design, remember to test throughout the process – from paper sketches to high-level prototypes. That lets you catch issues early.
cindy.mccracken
2016-11-30 23:10
For recruiting, consider who your target users are and where they are, then ask screener questions if you need to be specific
cindy.mccracken
2016-11-30 23:10
The test plan should include all the details of the study, such as the goals, session information (such as how observers can log in), all the tasks, etc.
cindy.mccracken
2016-11-30 23:11
When moderating, of course follow the test plan. Remember to stay neutral in words and actions; make participants feel comfortable; and remind people to talk if they’re not so you can learn.
cindy.mccracken
2016-11-30 23:12
Finally, with findings and presentations, prioritize your recommendations (low, medium, high), and definitely meet with stakeholders in person
cindy.mccracken
2016-11-30 23:12
In person is important so you can make sure the results are being interpreted correctly, and also so you can discuss results with business goals, etc.
cindy.mccracken
2016-11-30 23:13
Remember that in usability testing you are finding problems, not solutions.
cindy.mccracken
2016-11-30 23:13
So don’t get into talking about solutions at your debrief session.
nat
2016-11-30 23:14
There are many types of testing that you can do, do you have a key few you tend to go for first?
cindy.mccracken
2016-11-30 23:14
These are all high-level ideas. I’m curious why you’re here today … what you’re interested in.
cindy.mccracken
2016-11-30 23:14
Yes, I think moderated testing – either in-person or remote – is the best first way to get qualitative feedback
cindy.mccracken
2016-11-30 23:14
You can get great qualitative results about “why” your designs are and aren’t working well.
cindy.mccracken
2016-11-30 23:15
You can do a cafe study to be even quicker.
jacqui_dow5
2016-11-30 23:15
Do you find the findings in remote testing and in person testing give you the same validity?
cindy.mccracken
2016-11-30 23:15
If you do a cafe study, it should be very brief – like 3-5 minutes.
cindy.mccracken
2016-11-30 23:15
Just ask one or two most important tasks.
cindy.mccracken
2016-11-30 23:16
I have found remote testing to be very useful … a lot of times it’s been the only way I could get the right participants.
jacqui_dow5
2016-11-30 23:16
We are having the same
cindy.mccracken
2016-11-30 23:16
The main thing you miss is seeing people’s expressions. Try to make sure they’re talking a lot.
cindy.mccracken
2016-11-30 23:17
You can try to get people to share their faces with their webcams, but sometimes that can feel invasive or awkware
cindy.mccracken
2016-11-30 23:17
awkward
jacqui_dow5
2016-11-30 23:17
We’ve had it where multiple people have been doing the test at once, should we try and prevent this?
cindy.mccracken
2016-11-30 23:17
Yeah, remote is a great way to get more access to people. The incentive can be lower too.
cindy.mccracken
2016-11-30 23:17
You mean two or three people at a computer.
jacqui_dow5
2016-11-30 23:18
Yes! We had one a few weeks ago where we think there were 4 people in the same room around the PC
cindy.mccracken
2016-11-30 23:18
It can just get confusing … but I’ve had that happen too when it was a very import person who wanted others involved.
mpcnat
2016-11-30 23:18
How do you handle usability testing of a mobile app that is being tested by a remote user, and due to logistics and comms issues the facilitator is an account manager from your own company. What tips do you suggest to get some success out of the process?
jacqui_dow5
2016-11-30 23:18
I was worried if one was a ‘boss’ it may influence the others
cindy.mccracken
2016-11-30 23:18
You can try to get them to do it separately if it makes sense
cindy.mccracken
2016-11-30 23:18
Or if it doesn’t, ask that just one person be in control and talking.
lynne
2016-11-30 23:19
Can you explain what you mean by a cafe study? I can make a guess, but it’s the first time I’ve heard this term…
cindy.mccracken
2016-11-30 23:19
so … the remote testing is in person, but the AM is the facilitator?
cindy.mccracken
2016-11-30 23:19
oh wait, remote
cindy.mccracken
2016-11-30 23:19
how is it being done remotely? what method?
cindy.mccracken
2016-11-30 23:20
@jacqui_dow5 – that’s why if they could do it separately it would be better.
cindy.mccracken
2016-11-30 23:20
it’s like in a focus group … where one person might take over
hawk
2016-11-30 23:20
Note: I’ll acknowledge questions that have been queued with a :grey_question:
cindy.mccracken
2016-11-30 23:21
@lynne – a cafe study is where you set up in a cafeteria, or at a mall or somewhere with your design.
cindy.mccracken
2016-11-30 23:21
You have an intro and a task or two for people.
jacqui_dow5
2016-11-30 23:21
Is cafe study the same as geurilla testing?
cindy.mccracken
2016-11-30 23:21
Then you have people do the task on your computer or device, and pay them with a small gift card or something.
cindy.mccracken
2016-11-30 23:22
It’s just usually to get quick feedback on a particular interaction.
cindy.mccracken
2016-11-30 23:22
yes
kaydeecarr
2016-11-30 23:22
What is your technique for analyzing the data after you’ve done the testing?
cindy.mccracken
2016-11-30 23:22
@kaydeecarr – I spend time making sure I take clean notes that are organized and will be easy to analyze.
cindy.mccracken
2016-11-30 23:22
columns for participants / rows for questions.
cindy.mccracken
2016-11-30 23:23
Then I go through and add rows to count the way people behaved (accomplish task, etc.)
cindy.mccracken
2016-11-30 23:23
and add those (this is all in excel)
cindy.mccracken
2016-11-30 23:23
Also, just keeping track of big issues related to goals as we go through the test sessions.
cindy.mccracken
2016-11-30 23:23
Then at the end go through to find data to back up those findings.
cindy.mccracken
2016-11-30 23:24
but I go through everything in case I missed something.
cindy.mccracken
2016-11-30 23:25
@mpcnat How do you handle usability testing of a mobile app that is being tested by a remote user, and due to logistics and comms issues the facilitator is an account manager from your own company. What tips do you suggest to get some success out of the process? (remote participant, local account manager facilitator)
cindy.mccracken
2016-11-30 23:25
just repeating this question.
cindy.mccracken
2016-11-30 23:25
what is the process? can you see the device?
cindy.mccracken
2016-11-30 23:26
are they using webex on an ipad? something like that?
cindy.mccracken
2016-11-30 23:26
or holding their hands in front of a laptop’s webcam?
cindy.mccracken
2016-11-30 23:26
@mpcat – what issues / concerns are you noticing with the facilitator being an AM?
jacqui_dow5
2016-11-30 23:26
I’ve seen examples of where people attach a make shift camera holder to a mobile to record the screen? But would rely on the user making this and having a camera!
nik
2016-11-30 23:27
Is there a big difference between reports/presentations that are based on moderated VS unmoderated tests? It’s my impression that most insights from unmoderated tests comes from direct quotes whereas moderated tests include a lot of other factors, such as facial expressions, body language etc. And how does this affect the conclusions of a study? It seems to me that moderated tests will depend more on “personal interpretation” which can be more difficult to support scientificly.
cindy.mccracken
2016-11-30 23:27
Huh! That sounds interesting.
lukcha
2016-11-30 23:28
Mr Tappy!
cindy.mccracken
2016-11-30 23:28
@nik – In unmoderated tests, using UserZoom or Loop 11, I think most of the results are quantitative
mpcnat
2016-11-30 23:28
It’s a prototype app on a mobile device that the AM has in front of the customer, they have a facilitation script to assist them, that they take notes against whilst the customer performs tasks. The issue is that an AM isn’t familiar with usability Testing, but is great with the relationship, and they feel out of their depth, so do we cut down the tasks to test, but then then that means we lose insight with that customer, that we only see on an irregular basis
cindy.mccracken
2016-11-30 23:28
You would do them if you wanted greater certainty that certain things were problems, or to compare two different designs quantitatively.
cindy.mccracken
2016-11-30 23:29
So I would expect the results to be more charts, that sort of thing.
cindy.mccracken
2016-11-30 23:30
@nik – I don’t know if I’m answering your question. But in moderated testing, you still have findings – like whether people could complete tasks, where they got stuck, etc. So you have real facts to report. In addition to frustrations.
cindy.mccracken
2016-11-30 23:31
@mpcat – Is it in person then? It would help if someone else were taking notes so the AM could focus on facilitating.
jacqui_dow5
2016-11-30 23:31
I have used whatusersdo for unmoderated it was good as the users had a webcam so you got a similar video to moderated testing
cindy.mccracken
2016-11-30 23:31
It gets to be a lot to juggle.
cindy.mccracken
2016-11-30 23:32
Even better, have the AM introduce them to a UXer and have the UXer conduct the test. It can get overwhelming.
cindy.mccracken
2016-11-30 23:32
@jacqui_dow5 – Very cool! I’m going to take a look at whatusersdo. Thanks!
mpcnat
2016-11-30 23:32
yep it’s in person, sometimes there is someone else taking notes, but the feedback from the AM is “can’t you just give me the top 10 things you want to see, there is too much to watch out for”, which I get but since we may only get one go at it with them, it seems like we miss out, the script has tasks and also things to watch for (reactions) from the user on particular screens, this is where they feel overloaded.
cindy.mccracken
2016-11-30 23:33
Have you tried http://usertesting.com? You can create a mobile test for their testers to do on mobile.
hawk
2016-11-30 23:33
If you’re new to these sessions, you can jump in with questions at any time – they don’t have to be in context.
kaydeecarr
2016-11-30 23:33
Do you have any suggestions on what to do when users ask you questions? Like “how do I get to the homepage”?
robbin
2016-11-30 23:34
What’s a good way to practice writing up some questions? I’m new, so my worry is that I’m accidentally going to ask some leading questions.
cindy.mccracken
2016-11-30 23:34
@mpcat – Yeah, that’s definitely a lot to manage. When others take notes, and he doesn’t have to worry about writing down all the reactions, that doesn’t help?
cindy.mccracken
2016-11-30 23:35
@kaydeecarr – You can remind them that you’re there to see how they would do things if they were in their own environment. Ask where they think it would be … or where they would expect to find it
mpcnat
2016-11-30 23:35
It helps but sometimes it’s an AM by themselves on a site in the back of remote QLD :slightly_smiling_face: Thanks for the answers
lynne
2016-11-30 23:35
@mpcnat – can you get the AM to do a practice run with other people in the office? Might make them feel more comfortable with the task.
cindy.mccracken
2016-11-30 23:35
@robbin – definitely don’t ask users what they want … they don’t know.
mpcnat
2016-11-30 23:35
@lynne yep that’s how we are going forward with it. Thanks
cindy.mccracken
2016-11-30 23:36
know what important tasks for people to accomplish are … and ask tasks around that. be specific – like find a vacuum cleaner that gets good ratings and is in a particular price range.
cindy.mccracken
2016-11-30 23:36
What questions are you worried might be leading?
cindy.mccracken
2016-11-30 23:37
also you can follow up each task with a rating question – like how easy or difficult was that task (on a scale)?
robbin
2016-11-30 23:37
When I was practicing with the UX team, I was really tempted to ask things like, “But how do you think you would go back to the homepage?” and when I listened in, I also heard things like, “What do you think this button would do? Go back to the homepage?”
cindy.mccracken
2016-11-30 23:38
Try to keep them focused on accomplishing the tasks. If they’re looking at a button and not commenting, you can say, “What are you thinking?”
cindy.mccracken
2016-11-30 23:39
You’re right – that was leading. :slightly_smiling_face:
cindy.mccracken
2016-11-30 23:39
That is tempting … just try to stay calm and think before you speak. It’s hard to have restraint. I know.
nik
2016-11-30 23:39
You said “prioritize your recommendations”. What do you base this type of evaluation on? Especially, as an external consultant. Do you need access to certain data (e.g. visitors and clicks on specific websites or similar)?
robbin
2016-11-30 23:40
That’s what I thought :smile: I also wondered – how do you help the user feel a little more at ease? When I watch people, sometimes they feel really sheepish and I’m unsure if they’re being truthful. I’ve also been the tester before, so I’m used to saying exactly what’s on my mind because I personally don’t feel weird about it.
cindy.mccracken
2016-11-30 23:40
@nik – that’s a good question. I’ve always had a guide as to what makes something a critical issue vs. high or low, etc.
cindy.mccracken
2016-11-30 23:41
After this session, I can come up with something for you.
cindy.mccracken
2016-11-30 23:41
@robbin – Just try to relax and be friendly, and have a little chitchat at first. And definitely start with easy background questions.
cindy.mccracken
2016-11-30 23:41
Maybe offer them a drink. something like that.
cindy.mccracken
2016-11-30 23:42
@robbin – why don’t you think they’re being truthful?
cindy.mccracken
2016-11-30 23:42
do you think they’re trying to please you?
cindy.mccracken
2016-11-30 23:42
(that was leading)
robbin
2016-11-30 23:42
Hahah! Yeah, I think so – they sort of keep looking at me as if to ask, “Was that right?”
nik
2016-11-30 23:42
@cindy.mccracken That would be very interesting to see! I find myself struggling a bit with this and sometimes spend too much time on issues that might not be very critical
cindy.mccracken
2016-11-30 23:43
@robbin – Yes, that does happen! People think there are right answers. So you can definitely remind people that there really are no right or wrong answers.
cindy.mccracken
2016-11-30 23:43
… that you’re testing the software, not them.
hawk
2016-11-30 23:43
@cindy.mccracken I’d be interested in hearing about some ways to present your findings
cindy.mccracken
2016-11-30 23:43
You want to make sure the design is going to work for them, so you want their open and honest feedback. Really.
cindy.mccracken
2016-11-30 23:44
@hawk – I have done presentations – or results – several ways. One way that didn’t work well in my environments was a long report.
cindy.mccracken
2016-11-30 23:44
I don’t recommend that.
cindy.mccracken
2016-11-30 23:45
I tend to come up with a template for a PowerPoint report, and use that. I love using images and things like callouts to point out findings in the interface.
cindy.mccracken
2016-11-30 23:45
Sometimes color coding helps people see good vs. bad findings too.
cindy.mccracken
2016-11-30 23:45
I’ve tried bulleted lists in email, which can sort of work for agile.
cindy.mccracken
2016-11-30 23:45
But no matter what, it’s critical to present results in person, even if it’s in a casual setting.
jacqui_dow5
2016-11-30 23:45
We’ve had an issue where we are redesigning a system. Our current users have some preconceptions (we are hard to use, so things long winded) so when they test the new software it throws them when something is easy to do and they’ve started doubting themselves thinking we are tricking them! Any way of handling this?
cindy.mccracken
2016-11-30 23:46
You want to make sure everyone’s interpreting correctly and is on the same page about the findings – and agree on how they’ll proceed.
cindy.mccracken
2016-11-30 23:47
@jacqui_dow5 – Tricking them? I’ve never heard of that. I would just try to explain at the beginning of the test that you’ve heard issues and you’re redesigning because of them.
cindy.mccracken
2016-11-30 23:47
Maybe that will help their expectations? Or did you already try that?
jacqui_dow5
2016-11-30 23:48
Yes one comment was along the lines of ‘it’s a lot harder normally, I feel like I need to do more’
hawk
2016-11-30 23:48
There is 10 mins left in the session. If you’re sitting on a question, now is the time to ask!
jacqui_dow5
2016-11-30 23:48
We explain at the start why we are doing it and that this will be new and improved on what they’re used to!
nik
2016-11-30 23:49
Is it important to include “good findings” as well as “bad findings”? Or is it good enough to focus on problems in a report
cindy.mccracken
2016-11-30 23:49
@jacqui_dow5 – interesting to take note on that. that could be a good quote. Oh – which I forgot to mention … I love quotes in presentations! They can really be effective at conveying attitudes.
cindy.mccracken
2016-11-30 23:49
@nik – I actually always try to do both. Definitely.
cindy.mccracken
2016-11-30 23:49
The designers have worked hard on the product, and also they just need to know what IS working well so they keep those things.
cindy.mccracken
2016-11-30 23:49
It shouldn’t be all doom and gloom.
nik
2016-11-30 23:51
@cindy.mccracken That makes sense, I’ll keep that in mind. I guess including some bright sides will also make the team more inclined to want to involve you in a project again some other time :slightly_smiling_face:
bleke
2016-11-30 23:51
@cindy.mccracken Steve Krug talks a lot about getting decision makers to view the sessions. Is it something you normally do?
cindy.mccracken
2016-11-30 23:51
@jacqui_dow5 – Huh. Maybe follow up their – I need to do more – with a question. “Why is that?” or just get them to elaborate so you can learn more from it.
cindy.mccracken
2016-11-30 23:51
@bleke – I’m glad you brought that up!
jacqui_dow5
2016-11-30 23:52
Yes, good idea! It completely threw up in the test!
cindy.mccracken
2016-11-30 23:52
Yes, definitely have observers – as many as possible. It really does help with buy-in and general understanding of the problems when people observe.
jacqui_dow5
2016-11-30 23:52
Threw us*
cindy.mccracken
2016-11-30 23:52
You can simply share your screen and have people log in to the screen-share tool. Have them ask questions of the note-taker or through a different channel so participants aren’t distracted.
cindy.mccracken
2016-11-30 23:53
@jacqui_dow5 – I wondered what that meant.
jacqui_dow5
2016-11-30 23:53
Sorry it’s getting late here!!
cindy.mccracken
2016-11-30 23:53
@bleke – Definitely encourage people to come to as many sessions as they can attend.
cindy.mccracken
2016-11-30 23:53
@jacqui_dow5 – That’s OK! I’m glad you came!
bleke
2016-11-30 23:54
@cindy.mccracken Ok, thanks! :slightly_smiling_face:
lynne
2016-11-30 23:54
I recently attempted some usuability tests in a classroom setting, with multiple high school students using our site at once. It was very hard to manage – we got some good data, but I’m wondering if you have any suggestions on best practices for a session like this?
cindy.mccracken
2016-11-30 23:54
Re: observers – have a few minutes at the end where you ask observers if they have questions, and then ask them of the participant.
cindy.mccracken
2016-11-30 23:54
You don’t want to be interrupted by observers.
cindy.mccracken
2016-11-30 23:54
@lynne – was there a reason for multiple students using the site at once?
cindy.mccracken
2016-11-30 23:55
Like, is that how they normally use it/
cindy.mccracken
2016-11-30 23:55
?
nik
2016-11-30 23:55
One last question from me: Often management views usability testing as a box that should be ticked at the end of product development. Any advice for how to convince management that it should be done as early as possible in a project?
nik
2016-11-30 23:56
I’m hoping there’s one Magic Argument that always works
cindy.mccracken
2016-11-30 23:56
@lynne – It is hard to capture exactly who’s saying what. But if it’s necessary for multiple to be using, maybe focus on their interactions and how they’re doing it.
lynne
2016-11-30 23:56
Yes, it is used by schools in class. But the main reason was because we had an opportunity to do so and it’s hard to get into schools so we tried to make the best of it.
frankenvision
2016-11-30 23:57
Is it ok to make changes after a single usability test? If they are obvious blunders?
cindy.mccracken
2016-11-30 23:57
@nik – hmmm… I wish that too! More or less, do usability testing early as often as possible – even if very simple, affordable ones – to prove how it works and how important it is.
cindy.mccracken
2016-11-30 23:57
In other words, show don’t tell.
cindy.mccracken
2016-11-30 23:57
That worked really well for me at one company where they weren’t convinced of the value until we proved it.
cindy.mccracken
2016-11-30 23:58
If not that, find some good case studies to share …
cindy.mccracken
2016-11-30 23:58
And a lot of data that shows starting early is a best practice. If you wait too late, you can’t do anything with the data.
cindy.mccracken
2016-11-30 23:59
@frankenvision – Usually you will read that you shouldn’t make changes to design … but I think it’s OK if it’s something that’s not working but should have been, and you’re having to talk around it … that sort of thing.
cindy.mccracken
2016-11-30 23:59
That’s not a design change then; it’s fixing a bug.
robbin
2016-12-01 00:00
(Just wanted to say thank you for doing this! This is super helpful!)
cindy.mccracken
2016-12-01 00:00
@lynne – so you were trying to get a lot of kids’ feedback at once to make the most of your time? Here’s one idea … you could have them use the program individually .. and observe as much as you can. Then have a focus group of sorts where they discuss.
cindy.mccracken
2016-12-01 00:00
It’s a thought!
cindy.mccracken
2016-12-01 00:00
Thanks, @robbin! I’ve really enjoyed it.
frankenvision
2016-12-01 00:00
Ok thanks – how many questions should we use for remote usability tests? I think they’re supposed to take 15min for users to complete…
frankenvision
2016-12-01 00:01
What a good question to measure how trustworthy a company is?
cindy.mccracken
2016-12-01 00:01
It might depend on how long it will take to do the tasks. I’d say I’ve seen about 10 tasks on average. Best idea: test it with some people to see how long it takes before sending it out.
lynne
2016-12-01 00:01
That’s kind of what we did, they each had a computer, we gave them tasks to do, and discussed as a group at the end. They were a bit shy about the discussion part.
cindy.mccracken
2016-12-01 00:02
One idea for shyness that I like is first having them write down answers to your questions.
lynne
2016-12-01 00:02
The biggest challenge was trying to observe 20 different people at once. I couldn’t figture out if it was better to choose one or two students to focus on, or roam around the room.
cindy.mccracken
2016-12-01 00:02
Then, having thought it through first, people tend to be more confident.
lynne
2016-12-01 00:02
Nice idea about writing it down – i’ll try that next time!
cindy.mccracken
2016-12-01 00:03
great!
cindy.mccracken
2016-12-01 00:03
@lynne – maybe if that happens again, it could help if you had multiple observers??
cindy.mccracken
2016-12-01 00:03
that’s a tough one,.
hawk
2016-12-01 00:04
Ok all – after this answer that’s a wrap
frankenvision
2016-12-01 00:04
Is it best practice to run a single test to see how it runs or run 3-5 remote usability tests first go?
hawk
2016-12-01 00:04
Remember that if you have follow up questions you can ask at http://community.uxmastery.com
hawk
2016-12-01 00:04
Someone is there pretty much around the clock
seyonwind
2016-12-01 00:04
Thank you for all the usability advice @cindy.mccracken!
As always, thank you @hawk for hosting :slightly_smiling_face:
As always, thank you @hawk for hosting :slightly_smiling_face:
cindy.mccracken
2016-12-01 00:05
Thanks everyone! I really enjoyed this.
jacqui_dow5
2016-12-01 00:05
Thank you so much guys! This has been great!
hawk
2016-12-01 00:05
Thanks so much again for your time @cindy.mccracken – you rocked it!
lukcha
2016-12-01 00:05
This has been a great session_thanks so much for your advice and tips @cindy.mccracken :slightly_smiling_face:
cindy.mccracken
2016-12-01 00:05
Of course. And thank you for hosting,
lynne
2016-12-01 00:05
Thanks @cindy.mccracken and @hawk!
hawk
2016-12-01 00:05
And thanks for all the great questions
hawk
2016-12-01 00:05
Have a great morning/afternoon/evening/night all
cindy.mccracken
2016-12-01 00:05
Indeed.
nik
2016-12-01 00:06
Thanks a lot @cindy.mccracken . Good night
nat
2016-12-01 00:08
Thanks!
lukcha
2016-12-01 00:12
Thanks everyone!
Excellent session. Cindy is a master!