We’ve found that a lot of first time UX researchers have similar questions and concerns when they start working in UX design. So, we thought we’d round up and tackle some of the most common questions to form a set of useful principles for UX researchers. Of course, this isn’t a complete guide to UX research (there are some fairly weighty tomes out there which do that) but it’s a good starting place to answer some of those nagging UX questions.
Mix It Up
The best researchers don’t use a single tool to do their research. They take a host of different methods, tools, etc. and then mix them together. This gives you a much greater chance of finding the actual issues and then being able to fix them.
It’s Easier To Find “You Got it Wrong”
Research can quickly help us work out when we’ve got something wrong. If you add a new feature and your first five research participants hate it – there’s probably a problem. However, a hundred people can use something without comment and it may still not work.
You Can’t Standardize Sample Sizes For All Your Research
Sorry, but sample sizes need to be calculated based on the risks you’re willing to assume in any given piece of research and based on the type of research you’re carrying out. Don’t try and use a single size for all of your research – it’s a flawed approach.
Testing With Just One User is Not Pointless
Imagine you’re creating a new word-processing package and you sit down with your first user and they try to save a document and you see that the process is broken. How many more users do you need to test that with? None, right? Some problems are universal and it only takes one user to point them out.
Author/Copyright holder: Luca Mascaro. Copyright terms and licence: CC BY-SA 2.0
Increase Sample Sizes for Better Accuracy
The bigger your sample size, the more likely your data is to be accurate. There’s a general rule of thumb that says to double the accuracy you have to increase the sample size by a factor of four!
Randomizing Can Overcome Research Design Flaws
If you can change the order of questions, responses, process flow, etc. then do it. The more random the path you take – the more likely it is that you’re going to get consistency and minimize experimental design flaws.
Research Results Belong to No-One
All that data you’re collecting? It’s not yours. It’s not your team’s. It’s the company’s. The more you get your user experience research out into the company as a whole – the more likely it is that your company will start focusing on user needs as a priority. Don’t create a UX silo in your business; let the data flow and reap the rewards.
Scale Ratings In Questions Aren’t That Important
Sure, there are plenty of arguments about whether an x-point scale is more accurate than a y-point scale and whether you should have a neutral rating or not. None of them are important enough to spend more than 5 minutes worrying over – pick a scale and do the research already.
Participants Need to Reflect Personas
Not everyone is a user or even likely user of your product. Not every user fits your target market. Get your user personas out and recruit to the persona – that way you have the most chance of getting results that actually work for your target users. You can’t please all the people all of the time and UX professionals shouldn’t even try to.
Author/Copyright holder: Nicolas Nova. Copyright terms and licence: CC BY 2.0
What They Say vs. What They Do
It’s often said that what people do is what matters and not what they say. We don’t agree. You need to measure both what people say and what they do. Then you can explore the reasons for the disconnection between the two positions. Sometimes people really do want what they say they want and sometimes they don’t. Knowing when those things are true matters to the user experience.
Keep Growing Your Toolkit
There are going to be new ideas and methods floating out there for a long time to come. Don’t dismiss them without trying them. Even if they suck, you’ll have learned that they suck rather than assuming it. In many cases even the worst tools can offer reasonable insights if they’re adapted properly.
Usability – A Polite Fiction?
It is impossible to measure usability. What we can measure is when something is not usable. Those measures are fluid – they change from product to product, user to user and UX researcher to UX researcher. That’s OK, finding problems is part of what research is about. We already know it’s much harder to show there is no problem than to find a problem.
Keep Reports Short
Sure, the method was amazing and innovative and the results were incredible but… you don’t need to write a book to get that across. If you want your research to have wide value in the organization keep your reports to a minimum. However, don’t let that stop you from creating more detailed work as a learning tool within your own environment or from writing that book if you intend to publish it commercially.
Be Aware that Observers Observe Differently
There’s a reason police treat eyewitness testimony with a certain healthy scepticism. People see what they’re going to see and rarely will witnesses see the same things. That’s not a major problem; in fact, it means that adding observers may increase the overall success of research – if you all identify different problems, that’s better for the users (as long as you intend to fix all those problems, of course).
And don’t forget that the act of observation may also change the results that you get.
Cults of Personality Suck
There are a ton of UX gurus out there. Some will be highly trendy today and treated with contempt tomorrow or vice-versa. There’s no one “right” way to be a UX researcher; ignore the name attached to UX ideas and focus on the underlying idea instead and treat everything with a healthy dose of scepticism and interest.
Header Image: Author/Copyright holder: Zeke Franco. Copyright terms and licence: CC BY-NC-ND 2.0
