It’s an important question. Can we measure the user experience and determine whether the design is good? Or are we championing a lost cause without knowing it? In order to be taken seriously as UX professionals, we need to be able to show that our work has meaning, and that means having something tangible as evidence that we have something valuable to bring to our organizations.
Can You Measure Experience?
The simple answer must be yes. We can measure experience because we must measure it. How else do we know if our work is doing the job correctly? We need data to show that we’re heading in the right direction and to show what we’ve done so far on our journey. Ideally, that data should be a reflection of the objectives of our business, be they financial, satisfaction, or something less obvious like safety.
The measurement process must provide insight into the people using our products. It should deliver a greater level of understanding than we had before. So, for example, we might measure the bounce rate on our website. It tells us whether users are sticking around more or disappearing faster than before we made a change. A single measurement won’t give us a perfect view of whether the design is great or not, but you don’t have to use a single measurement to measure something.
Can You Measure Experience Easily?
The truthful answer to this is no. Building an accurate set of measurements for our design work is not a simple process. The easier a measure is to make, the more likely it is that we will be measuring something irrelevant. If you go back to the early days of social media work – everything was measured in likes or follows. Marketers have wised up to the fact that these metrics are pretty meaningless now. You must dig deeper, which requires more effort if you want something of value.
Can You Measure Experience Accurately?
Probably not. Experiences change over time. We have all abandoned products, applications, and websites that we once loved because they’re no longer relevant. However, that doesn’t mean that we can’t measure with some degree of accuracy either. It also doesn’t mean we can’t change our measurements to reflect business or user changes.
You don’t need to promise easy or perfect. As a UX pro, you need to develop sensible metrics that at least reflect what you’re trying to do and your understanding of your users.
How Do We Measure the User Experience?
There’s no one-size-fits-all approach to this, and what you do in your organization will differ from what we do at the Interaction Design Foundation. However, there are some common principles to consider when choosing your metrics:
Don’t Be Afraid to Ask a Single Big Question
Barometers don’t predict the weather but give us a decent indication of what might be coming. Building a barometer for the user experience of your product is a simple way to see which way the wind is blowing.
People aren’t good at remembering fine details. Most of us would need several minutes to explain why a film or book is our favorite if quizzed about the exact events within… we might need much longer. Many of us would find it almost impossible to get it right. But all of us can explain how we feel about something when we’ve just used it.
So why not ask a group of users this question; “How do you feel about our product?” Provide a rating scale or something similar so they can indicate that and you have a barometer. People who feel positively about your offering are more likely to buy or recommend you to others. Those who feel negatively are much less so.
Learn to Seek Objective Data for the Big Question’s Answers
The bigger the sample you draw on, the more likely the big question reflects an accurate picture of how your users feel about your product. You don’t want to stop with that question – there are plenty of ways to find underpinning data to see why people feel like that. Analytics data, eye-tracking data, etc., can all give you a better understanding of why the user experience is great or might not be so great.
Think About Meaningful Ways to Observe the Big Question Being Answered
Think about this for a second: does a sell-out at the box office when you see a film automatically make that film a good one? Most people who sat through The Phantom Menace (the first Star Wars prequel) can tell you it doesn’t. Sales don’t equal experience.
You might have wanted to stand outside the cinema and listen to how people talked about the film. You might have wanted to be in the cinema to listen to people boo at Jar Jar Binks (a character in the movie).
Your choice of measurements should be meaningful to the experience that you provide. Don’t rely on off-the-shelf popular techniques unless they’re truly relevant to your experience.
Take More than One Measurement
If you were to measure the experience of users of a social network that had just gone offline for an 8-hour maintenance routine – you’d find plenty of annoyed users. But would that accurately reflect how they felt about the social network? Of course not.
You need to measure experience regularly and look at trends over time. Don’t fall into the trap of treating measuring as a one-off exercise. Your snapshot in time may not be remotely accurate.
Develop a Dashboard of Metrics
Once you have the right metrics, put them together as a dashboard and have an “at a glance” feel to the current user experience. This will help you communicate the value of what you do more easily to your organization and help you take action when the time is right.
Summary
No one said measuring experience was easy or a perfect science. Don’t be afraid to ignore data when it doesn’t appear meaningful, but do create data and constantly review how you use it – to give strong indicators of the user experience as a whole.
Image Sources: UX Refresh (link to image), Kate Rutter (link to image), Heather McQuaid (link to image), Igda (link to image), Roxanne Cook (link to image)
