DATE

September to December 2019

ROLE

Author, Recruiter, Interviewer, Moderator, Designer

DESCRIPTION

A user reserach project focused on understanding why students subscribe to ClutchPrep

I wanted to understand what influenced Clutch subscribers' overall decision-making. So, I conducted user research and compiled a timeline of Clutch customers' experiences and decisions that caused them to subscribe. The research process for creating this timeline involved:

Defining the problem
Selecting the approach to collecting research
Planning and preparing to research
Collecting data (interviewing users)
Analyzing data
Reporting the results

1. Defining the Problem

I collaborated with my manager and an engineer to define the problem this research would seek to solve.

A mind-map of the user research process and defining the problem

#2 Selecting the approach

We used the Jobs to be done method to create a timeline of users' decision-making prior to subscribing. This method helps us understand what service failed to change their status quo and how our service helps us solve this problem.

Conveying the idea of jobs to be done with changes in status quo.

Image from leanstack.com

#3 Planning and preparing to research

This was the most extensive part of the process. Planning involved the following steps:

Analyzing existing data to understand how to select users
Creating an interview script
Creating a format for debriefing
Recruiting and scheduling interviews

Mind-map of deciding how to analyze existing data

The interview script begins with a warm-up and follows the idea of trying to understand progress. It breaks the conversation into smaller components that allowed us to see the whole picture.

I worked alone to create this script but received feedback from others involved in setting the goal. This feedback ensured the questions helped us to reach the final output

The debrief template focused on Jobs to be done. The outcome focused on understanding the old solutions that had stopped working for the users. This understanding encompassed the emotional and social aspects of the hire.

Recruiting and scheduling: This was probably the most difficult process, because our email open rate was around 25%. So, I texted users to recruit them for interviews and offered a $30 incentive per user ($50 for users whose usage patterns we particularly wanted to understand).

#4 Collecting data (interviewing users)

Interviews were one hour each. I was the point-person for scheduling and conducting interviews and delivering compensation. I wanted to make sure as many people from the leadership team and the decision-making process were involved. so I asked them to join the interviews as observers.

One of our engineers was the primary note-taker for all the interviews. He and other observers sent me questions during the interview if and when they wanted to learn more about a certain area or neede clarification.

#5 Analyzing data

Analyzing qualitative data is work-intensive but always results in surprising outcomes. For this process, we first annotated all our interviews and then broke our annotations into different categories (one point per Mural Post-It). We worked on this process remotely, so Mural was an excellent tool to create a collaborative canvas.

This was very much a collaborative activity with the engineer who worked as the note-taker and the manager who worked with us to set the initial goal for the research.

We then broke the interview annotations into Post-Its under four main categories:

Pushes
Pulls
Anxieties
Miscellaneous info

The pushes represent the reasons they felt the existing solutions didn't work well.

The pulls represent our product's features that allowed them to solve their problem.

The anxieties held users back from subscribing to our service.

Lastly, the miscellaneous info includes information we didn't initially seek but realized were useful.

#6 Reporting the results

Based on the analysis of all the interview data, I spent over a week working on the report and gathering feedback from all the stakeholders who observed some sessions.

The format of the report was:

Executive summary
Method
Findings
Recommendations
Appendix

One of the findings from the report.

Along with the report, I created a visualization to understand the user journey. I sketched it first to gather feedback about what the stakeholders wanted to see along the journey.

User Journey

Based on the stakeholders' feedback, I updated the user journey to show how students study and how they address their challenges.

Challenges

There were two main challenges

Recruiting users
Inviting stakeholders to interviews

To address the first challenge, I took a unique approach of texting users to ask them to participate in interviews. Many more users responded and were willing to share their feedback than when we had communicated through email. This along with changing the interview format from video to phone interviews encouraged many more users to participate.

An immense part of research is collaboration, and it was important for leadership and decision-makers to be exposed to and involved in the process. I asked my CTO about this, and he agreed to bring different stakeholders in for these interviews.

I created a clear interview schedule and shared it with the stakeholders who could join based on their schedules. I also made a new Slack channel to notify the stakeholders 15 minutes before an interview began. That way, if they had an unpredicted schedule opening, they could still join.

Learnings

This was a very extensive process that generated concrete results aligning with our expectations. It helped me understand the importance of collaboration during the research phase. Because stakeholders observed the findings firsthand as opposed to only hearing the final conclusions, they understood and agreed with the interpretations. Additionally, department managers had different ideas based on their particular domain expertise. As the stakeholders discussed the findings, we watched their excitement as they brainstormed how to incorporate the results into strategies for marketing, leadership, etc. Making this process more transparent helped unify and strengthen the team.