Every product manager’s dream is to launch a product or new feature that is perfect for their customer right out of the gate. But we know this is hardly ever the case.
It’s only through experience, and repeated user feedback using user experience testing, that we learn to tweak our product’s features and interface according to our users’ conscious or unconscious demands.
Over the last two decades, agile and sprint based development has enabled an efficient and effective product management – development team – design team feedback loop. The next stage in the Agile revolution is to add the customer and the user to this cycle, through Agile user experience testing (Agile UX Testing).
Test small, test often
Conducting market research and usability testing used to be an expensive and lengthy process. That’s why it was typically performed only once, or maybe twice during the design and development of new products and features. Since UX testing was often done towards the end of the product development cycle, the feedback gained was often used more as a “validation” exercise than an “exploratory” exercise.
It’s now possible to set up a UX test script in 5 minutes, receive qualitative picture-in-picture responses (webcam view recording of respondents + screen recording + audio + quantitative data) within hours, and add the customer and the user to the agile software development feedback loop.
You can even run a user test for every sprint, and for every design and development iteration. This iterative process avoids the risk of putting together a complete prototype and realising far too late that there is a flaw in the design that should have been dealt with in the early development stages
You don’t need to run these tests with large sample sets. It’s a rule of thumb within the usability industry that 5 participants going through a qualitative picture-in-picture recording session like that of Userlytics.com will allow you to uncover 80% of the usability and user experience issues in a design.
See who is testing your product
It’s important to always have your target persona front of mind. It helps you look at your product through the lens of your customer – as much as possible.
However, it’s impossible to fully immerse yourself as your own customer no matter how hard you try. If you can’t be them, watch them. The best feedback you can get on your product is from the users its meant for.
Online remote tests with only screen recording and audio don’t allow you to know if the person providing the feedback is truly your target persona. Including a webcam view of participants adds a whole new level of depth and details to your qualitative UX testing insights. You can start to verify whether the person is actually your target persona, and better understand their contextual surroundings as they interact with your UI and answer questions. By being able to visually analyse the tester you can track their real-time emotional/physiological reactions to the product.
When launching a test with Userlytics, you can use demographic filters and set screener questions to ensure their global participant panel provides you with a tester that fits your persona. They also track participant location via their IP, and review every result to reject or approve it through their QA process.
Getting feedback on your product is always great, but it’s even better when the feedback is from participants that match your target persona.
Moderated vs. unmoderated vs. hybrid
For lab-based user testing, moderated testing (a UX researcher guiding the respondent) used to be the norm. The advent of new technologies allowed UX researchers to conduct moderated sessions on a remote, online basis, using screen sharing platforms like GoToMeeting, Webex, Zoom, Skype, Hangouts and so on. The problem is scalability. UX design researchers only have so much time to moderate target participants, let alone conduct the project management and scheduling and logistics required to manage a moderated usability testing process.
When Userlytics and its peers in the industry invented unmoderated usability testing, the economics, time requirements and scalability of user testing advanced by orders of magnitude. The drawback? A rigid non-personalisable test script.
In other words, a single UX researcher could manage tests of hundreds or even thousands of respondents, anywhere in the world, at a fraction of the time, and cost required for moderated or in-lab UX testing. However, it was not possible to adapt the unmoderated test script according to the answers and actions of each respondent, which would be possible with moderated usability testing.
But that problem has now been solved, through conditional logic (“branching” or “skipping” logic). The most advanced UX testing platforms have a hybrid approach that marries the scalability, speed and economics of unmoderated user testing with the personalisation of moderated user testing by leveraging branching logic. When branching logic is applied to a given task/question in the test script, it redirects the tester to a new task/question depending on their response to the prior question or task. Through branching logic you can essentially replace the moderator’s function through creating a customised set of instructions to different testers depending on their actions.
Efficient analysis
In an agile sprint where time is short, you need to get to your analysis quickly. Rapidly and economically launching a user experience test is only part of the equation. Reviewing and analysing the results, and decision-making in a timely manner is the other half. If you launched a test with 30-minute sessions of 10 participants, you and your colleagues have 5 hours of video to watch, analyse, make annotations, and pull insights so as to enable product and UI optimisation. 100 participants would imply 50 hours. This time adds up quickly.
Your user testing platform should provide the tools you need to quickly review participant sessions, leverage searchable, time stamped & hyperlinked audio transcriptions to locate the most interesting actions and comments, add & share hyperlinked and shareable annotations, and create & share highlight reels.
Benchmark your product
Learning from how your users engage with your product is one way to ensure a positive customer experience. It’s also important, however, to see how your product compares to your competitors through benchmarking.
Benchmarking your prototype designs against each other, against existing production assets, and against your competition will allow you to identify additional opportunity areas where you can improve your usability design to create a superior customer experience.
The best way to do this is using pre-formatted system usability scale questions with automatic calculations of the resulting score. You can also use comparison metrics such as net promoter score, time on task, and success/failure, which allow you to quantitatively measure your usability and user experience against different design iterations and against the competition, or best practice websites and apps. “If you are not measuring, you are not managing”
TestFlight and Google Play equivalent
Testing your product early in the process is essential. However, once your mobile app prototype reaches a high fidelity stage and is on the Appstore or Google Play, you need to have a process for testing the apps prior to production.
Some platforms use testing apps, which may have the drawback of incompatibility with the tested app and the Testflight app. Others like Userlytics use an approach that does not require any kind of SDK or testing app, therefore avoiding potential usability testing pitfalls.
Conclusion
An agile UX testing approach takes an iterative customer-centric approach, providing many small sample tests with your target persona at every stage of the product lifecycle allowing you to optimize your product prior to launch. Like having your customers form a seamless whole with your sprint teams. This strategy reduces time and money wasted in design and development and results in a much higher customer engagement and satisfaction score.
If you want to learn more about agile UX, or how to launch an agile UX testing process you can schedule a demo here.
We just wanted to let you know this is a sponsored post. Every now and then, we partner with people and companies doing awesome things.
