Magitest: A Better Approach to Mobile User Testing

This is a review of Magitest, an iPhone app for conducting mobile user testing. The author was provided with a review copy of the app.

Summary

• Name of app: Magitest
• Platform(s): iPhone only; iPad, Android coming soon.
• Description: Allows you to perform mobile user testing of native apps and websites. Captures the participant’s voice and facial expressions during the session.
• Price: $24.99 (in-app purchase; free trial is time-limited to 60 seconds of recording time)
• Rating: 9/10

Here’s a video of me using Magitest with the browser pointed at uxmastery.com:

Full Review

A few years ago, I had an idea for an app.

I know, I know. Everyone has an idea for the next killer app, right? At the time, the app store was still in its relative infancy, and early stories of successful appreneurs were beginning to surface.

I’d been using Silverback, the simple but reliable user testing software from Clearleft (still my user testing tool of choice; see other similar tools). Silverback is basically just screen-capture software, but it cleverly makes use of the computer’s built-in microphone to record audio of the session, and the built-in camera to record video of the participant’s face, which is then overlaid onto the video in the corner of the screen. With Silverback, I’m able to conduct user testing sessions anywhere—at a client site, in a coffee shop, on a train—as long as I have my laptop with me. No need to book an expensive lab with a one-way mirror. Silverback has not only been instrumental for me in doing more user testing, but also in selling the idea to clients: “It’s so easy for us to run a test—why wouldn’t we?”

My idea was to create a “Silverback for iPhone”: an app that used the front-facing camera and built-in mic to turn the iPhone into a portable mobile user testing lab. It would be insanely useful for me, so I figured that other UXers would want it too.

Once I began to research the idea, I realised the technical challenges were entirely non-trivial, and I didn’t take it any further.

The Magitest team, on the other hand, have built just that app.

How I Used To Test Mobile Apps

Why is it so useful to have an app that lets you conduct user testing on your phone? Well, if you’ve ever conducted a mobile user test before, you’ll know that doing so is not straightforward. There are three basic options:

1. Use a tiny camera mounted to the phone with some kind of purpose-built rig.

   What I dislike about this approach is that any rig, no matter how lightweight or ingeniously engineered to hold the camera, is intrusive. Participants may find the extra weight of the rig disconcerting, aside from the visual distraction of a strange robotic arm protruding from the phone; this hurts our attempts to simulate a real-world scenario.

2. Use a video camera positioned to record over the participant’s shoulder.

   This is an approach I’ve had more success with—I’ll attach an inexpensive flip-cam attached to a monitor arm with rubber bands. While less intrusive, you need to instruct the participant to hold the phone at a specific height to ensure that the video camera captures the screen. There can also be issues with low audio as they are talking away from the camera.

3. Capture the user’s session on another device.

   It’s possible to use AirPlay to send a user’s session to another computer or AppleTV. I’ve also experimented with using an app called LiveView, which does the reverse and allows a user to interact with a desktop app (which can then be recorded on that computer). This approach can work well, but it doesn’t record any audio, is reliant on having a strong network connection, and isn’t an option for native apps.

Enter Magitest

Magitest is the app that I wanted to create. I’m not suggesting any appropriation of ideas or anything—the concept is kind of obvious, and execution is everything. But it’s exactly what I envisioned: Magitest is Silverback for the iPhone. If you’re a Silverback user, you’ll understand that this is a compliment: Magitest does one thing—record session data on your phone—and does it well.

The Pros

• Support for native apps.

  This is a huge selling point for Magitest. There are several apps out there that open a browser window and allow you to record the user session, but none of them work for native apps.

• No equipment required.

  As I mentioned above, being able to conduct guerilla user testing on a phone without a video camera, camera stand, phone mount, cable, or anything else is huge.

• No network required.

  Because Magitest performs all of its capture and post-processing of video on the device, there’s no worrying about whether your session will be lost due to network dropouts. It also opens up the number of places you can conduct user testing “in the wild.”

• Easy to use.

  Magitest is as easy to use as you might expect for an app whose premise is to encourage the user testing of mobile apps. There’s no hypocrisy here—no user manual required.

• Price.

  At $24.99, sure—Magitest is more expensive than the majority of games, productivity and novelty apps that dominate the Best Seller lists of the app store. However, when you consider the audience (people who perform user testing of mobille apps for a job) it’s a bargain (e.g. Silverback for Mac is $70).

• Aesthetic.

  Functionality aside, Magitest looks polished. It has a pretty icon, plays well on the magician’s aesthetic, and utilises typography that reinforces this branding message. All of this adds up to a more enjoyable experience.

That said, there are a couple of misgivings that I had about the app.

The Cons

• Speed

  The biggest hurdle I found with Magitest was that speed was an issue. It may be unfair to pin this on the app, as if you think about what is happening in parallel (screen render, screen capture, audio capture and video capture) then it is a big ask of your phone. I didn’t have an iPhone 5 handy to test Magitest out on, but I imagine that Magitest would be a lot zippier when testing both native apps and websites on the 4S or iPhone 5. If upgrading your phone is not an option, be prepared for your users to experience a bit of lag time while interacting with your app or website.

• Heat

  Somewhat related, I did notice my phone got a little warm after using Magitest for only a couple of minutes. This wasn’t necessarily a big hurdle, but it may be smart to use a plastic case that makes this heat increase less obvious to the user if you think it might become a distraction.

• Setup

  I didn’t get a chance to try Magitest out with a native app, mostly because the process requires that you export your app from Xcode. If you’re working closely with your development team, this is probably not a hurdle at all. However, it does mean you can’t user test production versions of your app, or do any competitive research by user testing other similar apps in your space. It may be a little unfair to list this as any kind of failing on the app’s behalf—it’s simply a necessity given the restrictions that Apple place on iOS apps.

Summary

Magitest is a great concept that has been executed well and I’m certain will change the face of mobile user testing. It is very demanding of resources, but I imagine this would be the case for any app that attempts what Magitest does. So while it was a passable experience on my iPhone 4, if I was going to be doing a ton of user testing for a mobile device, I would definitely upgrade my phone so that the performance was improved. There are also versions for iPad and Android in the pipeline, and I’m sure that this app will be very popular among UX designers.

Overall, Magitest definitely gets the thumbs up, and I intend to use it to test all of my mobile and web projects moving forward. I give the app a magical 9 out of 10.

Download Magitest from the iTunes app store.