Author, Interviewer, Analysis, Moderator
A usability test for a new product feature built on the test taking section of the website
Analytics was a new feature developed so expert subscribers at Clutch could better understand the academics areas they needed to improve. Our company had never worked on user-focused analytics projects. To ensure we were improving the interface as much as possible, we conducted usability testing.
Usability testing involves a series of steps:
This testing would focus on expert users. I determined the criteria with an engineer. These users have:
I created a simple email template to recruit these users. The incentive for users to answer our questions was a $10 Amazon gift card.
I opted to use calendly to schedule interviews, since it automatically creates calendar invites and reminded users before their interviews.
Setting up and running tasks
I broke down my usability test script into sections:
The test introduction ensured users understood the questions and tasks ahead. If they didn't feel comfortable, they could opt out of testing.
These are the high-fidelity wireframes that were tested for usability.
You can checkout the full prototype here
Analyzing the results
To analyze tests, I recorded each completed task as "pass" or "fail."
I combined the results with the debrief data and post-interview surveys to create a complete picture of usability testing. I also brainstormed findings and recommendations for solving the issues that came up.
I recorded each user interview so my teammates could watch according to their own pace. I compiled the findings and recommendations for changing the interface in a report distributed to other teams so they could benefit.
Left: I updated the chapter analysis to insert a scale indicating how many questions were correct and incorrect. I then repositioned it so it could be clearly understood.
Right: I added an option to reset exam review questions.
The biggest challenge was accomplishing usability testing within the allotted time. It took many meetings and presentations for different stakeholders to emphasize the importance of doing usability testing early. It would help us save time and challenge or affirm our assumptions of how users interacted with the interface.
Stakeholders ultimately allowed me to conduct usability testing. However, I had to run the process alone. This made it difficult later for stakeholders to accept my proposed changes.
Even though I was allowed to conduct the research, stakeholders did not want to be involved. Including at least two to three stakeholders would have removed any biases I might have had as an interviewer and designer and convinced them of the results.
My takeaway was to insist multiple stakeholders would be involved with usability testing and research.