Should We Introduce Benchmarking for our UX Research?

Benchmarking is beginning to get more attention as part of UX research strategies. However, we’ve found that many places conduct benchmarking as an exercise almost for its’ own sake with no deeper understanding of why they should benchmark. So let’s take a look at why benchmarking may be a good idea for your UX team:

Benchmarking – What is It?

A benchmark is essentially the output of continuously testing an application or website over an extended period of time. This allows you to track the progress of your product as you iterate development. It can also be used to compare your product to a competitor’s product.

Benchmarking – Why Do It?

The first benefit of benchmarking is that you have a clear idea of progress (or lack of progress) over a period of time. It provides insight in a quantitative manner that you often don’t get from UX research. Quantitative data (e.g. data you can measure in numbers) is easier to explain in many instances than qualitative data. It’s also much easier for people to challenge qualitative data; does that matter? Isn’t that just one person’s opinion?

The second benefit is that it can identify problems as they arise and also help you understand whether you have really overcome an issue.

Benchmarking – How Do We Do It?

As with all things UX you begin with a plan. You need to know what the purpose of a benchmark is before you create it. You should define precisely what you intend to measure. It is impossible to benchmark every aspect of a complex product; so you should focus your efforts on those areas that are most critical to your users and your sales.

You also need to ensure that you have the time, resources, money and commitment to conduct a benchmarking exercise regularly. There’s no point in setting a benchmark and then never referring to it again.

Once you have the commitment; you can start to develop the benchmark itself. You start by choosing tasks – based on a user’s objective. So if you were benchmarking for Amazon you might want to do; “Can I find this product easily?”

This also helps with developing descriptions of the benchmark that make sense to the rest of the organization. You want to be able to explain the value of the benchmark. “Put data into a box, and then click a magnifying glass” may be what you do – but it doesn’t explain why it matters.

You then need to focus on how you will get your users to evaluate this. You might choose a pop up box for example with the question; “Did you find what you were looking for?” with a Yes/No response. Or you might include the question in a user survey; “On a rating scale from 1 to 5 how easy was it to find the products you wanted last time you visited our site?”

Then it’s time to pick your users (make sure that you get a representative sample) and roll out the test. Don’t forget you need to do this regularly to make the benchmark frequently.

Header Image: Author/Copyright holder: Sonny Abesamis. Copyright terms and licence: CC BY 2.0