Perspectives Blog

Good research questions are the path to insights. Good research questions quickly and reliably place the participant’s thoughts, feelings, and behaviors under a microscope. Those insights can drive decisions — and keep your product moving forward.

Sometimes researchers ask questions that encourage research participants to give generic or idealized answers. In both qualitative and quantitative research, when you ask questions on “how things generally go,” you tend to receive responses that might stay high-level and therefore vague. 

‍

This approach might also influence participants to answer with an idealized version of themselves or with an answer full of platitudes.

‍

The best insights are in the details and you find the details by asking specific questions. 

‍

Let’s dig into two examples.

Example 1: You’re a researcher investigating how a participant pays their bills.

‍

You want to understand a specific instance, so avoid asking general questions like, “How is your usual bill paying experience?” or “Generally, how do you like your bill paying tools and how do you like them organized?” 

‍

They’ll likely answer with something vague. “Uh, I like it. Usually every month I just go into the website and pay the bill. It’s alright.” 

‍

But, do they really think it’s alright and a generally positive experience? Do they really ‘just go to the website’?  What was the inciting incident (a reminder email, a spouse or other householder, a to-do list, the power getting shut off)? How did they log in? How long did the task take? What else was pulling their attention at that time?

‍

Instead, ask the question in a way that brings up the most recent time: “When was the last time you paid a bill online?” After your interviewee has a second to reflect and call up their  last occurrence, they’ll bring up the specific memory and tell you about it. Then you, the researcher, can follow up with an open-ended “What happened…?” 

Example 2: You’re a researcher attempting to uncover TV viewing habits.

‍

Avoid asking, “How much TV have you watched in the last week?” Even though participants can come up with an answer, it’s unlikely that it’ll be accurate. They won’t go to any specific memory at all, but instead the participant will just produce a guesstimate.

‍

Often, they’ll imagine their average day and determine how much TV they watch on this mythical average day. Then, they’ll take that number and multiply by 5 or by 7 and confidently state their answer. If they imagine themselves watching 2 hours of TV a day, they’ll tell you either 10 or 14 hours. 

‍

But is that estimate accurate? Probably not.

‍

Also, they might ask themselves, “Am I a person who should be watching more or less TV than I actually do? Does a person who’s got these responsibilities and a particular identity — a teenager, a parent, a journalist, a busy CEO — watch TV as much as I do?” Whatever the interviewee decides from their moment of reflection can influence their estimate on how much time they spend watching TV. 

‍

Asking a general question will encourage interview subjects to fudge the numbers, but fudged numbers can be un-fudged by asking about specifics. 

‍

Try this question:

‍

“Thinking back to last night, did you watch any TV? What did you watch and how long did you watch for?” Or to get them to tell you how many episodes they watched - then you can calculate for them.

‍

Next, we ask, “Thinking back to the night before last, did you wind up watching any TV? What did you watch and how long did your session last?”

‍

In both examples, specific questions will uncover details that broad questions miss. 

Follow-up questions

‍

Specific questions aren’t enough. You need follow up questions. Two follow-up questions every researcher needs in their back pocket, “What happened…?” and  “Can you walk me through it… ?” 

‍

These open-ended kinds of follow-up questions allow interviewees to look back on their experience with a negative or positive lens. Looking back with a negative view, they might describe things with a somewhat negative bent, finding the points of friction or frustrations. It’s just as possible they’ll take the prompt positively, and give you a positive gloss on the experience. Asking ‘what happened’ allows the interviewee to shape their account as they wish. 

‍

The interviewer can also bring other lenses to the experience.

‍

“What’s different about the last time versus other times?” Now that the researcher has a specific memory or two in hand, she can get into the comparisons and reflections on multiple occasions or the ‘usual way it goes.’

‍

“How do you break down the activity or process in your mind?” When the participant gives their account, they’ll almost always break it down into steps. Asking explicitly helps get more in their individual mental model for the activity. 

‍

“What was the most difficult part of the activity?” If the interviewer wants to dig into the tougher parts or steps or where things went badly, this question is a solid starter to encourage a participant to re-cast the experience more negatively.

‍

“How do you think you will change your approach next time?” This clarifies what they learned and how they can apply it in the future. 

‍

The art of crafting research questions extends beyond mere generalities. Put some effort in your research questions, avoid making them general. Remember, the best insights come from good research questions and you want to avoid the pitfall which lies in inquiries that invite participants to offer commonplace or idealized responses. Both qualitative and quantitative approaches suffer when relying on queries about how things usually unfold, yielding vague results or polished self-portraits. 

‍

Consider the two instances: there are  intricacies of bill payment habits and the nuances of TV viewing routines that will be overlooked and missed if your questions are too broad. Ask questions that bring up a specific experience and then broaden your questions from there.

‍

Being specific isn’t enough - you need the follow-up questions that help you and the interviewee understand the experience more deeply. This all echoes the notion that the finest insights spring from well-crafted research questions, encapsulating the wisdom that fuels progress. As John Dewey aptly noted, "We do not learn from experience… we learn from reflecting on experience." So get people to reflect on their behaviors — in the reflection lies the insight.

‍