Usability testing: measuring UX success
How to build user scenarios, conduct a usability test, and measure the outcome.
It’s safe to say that all designers want their products to be useful and enjoyable. We could measure those things by understanding the users. How do they behave? Is it easy to use? Are they overwhelmed? Would they recommend it? Well, the most straightforward way to know the answers is to ask the users themselves.
Usability testing (UT) or user testing is a practice to measure the success of UX in a product by asking for direct user feedback. There are a lot of ways you can go through with it, some of them are surveys, interviews, and walkthroughs. It all comes down to what outcome you want though, so make sure to set an objective first.
In this article, I’ll mainly focus on covering the practice of UT for my current team project. You’ll see how I and my team put together a user scenario for the following UT interview, how to conduct the said interview, and later how we would analyze the results.
Prep
As I mentioned, before you do anything in this step, take time to set a goal. What are you looking for in this UT? Do you need both quantitative and qualitative results or just one of them? What are the available resources?
- Surveys are quantitative in nature, meaning the insights you get will be numerical. It can reach a big amount of people in a short time, but it doesn’t accommodate in-depth feedbacks.
- Interviews and walkthroughs are qualitative in nature, each respondent will require a certain amount of time to test, and you can understand the ‘why’ of their behavior.
Because our team wanted to understand user’s behavior and also measure the usability rate of the current design, we decided to use both qualitative and quantitative methods. Qualitative by in-depth interview and quantitative by SUS questionnaire.
Building User Scenario
To have a standardized and guided interview, we will make a user scenario.
Imagine writing a script where the actor is being an interviewer in a movie scene
Writing a user scenario is the same thing, but you are being the interviewer and the questions only revolve around getting the right feedback for your product. The general structure is as follows:
- Start with an introduction. Tell the user who you are, the objective of the UT, and describe a little about the platform you’re trying to test. A briefing basically.
- Gather their demographics. Ask them to introduce themselves. You can list all of the aspects you need, such as name, age, domicile, etc. Make sure it’s non-intrusive.
- Go through the motion. Write the instructions you’ll give to the user, step by step. Don’t forget to write down the questions you’ll need to ask for each of the tasks.
Here’s an example to give you an idea of what it’ll look like:
Conducting UT
The motion of UT is exactly as what you wrote in your user scenario. But before conducting any UT yourself, logically, you need to find respondents. The best practice is no more than 5 users because any more than that, the insights you’ll receive will most likely be the same as the insights from the first 5 users. Don’t forget to also pick respondents who are a match with your target user’s characteristics.
You can conduct UT online or offline, although take note that you can observe user’s physical behavior better by doing it offline. Again, adjust to what you and your team can afford.
Analyzing the Result
In analyzing the result, quantitative and qualitative data require different treatments. Qualitative insights from the interviews need to be reviewed and synthesized. Whereas quantitative insights from the SUS questionnaire can be analyzed by comparing the scores from each of the respondents.
What you will get out of the UT will be in-depth feedback on your platform’s user experience. This knowledge is crucial if you want your products to be useful, enjoyable, and up to date with the trend.
