Perspectives Blog
Why asking “do you like…?” is a really bad research question
Dive deep into meaningful insights: asking open-ended questions in qualitative research
There’s the old trick question that goes something like, “When did you stop shoplifting?” It’s impossible to answer that question without either saying that you at one time did shoplift or you continue to do so.
It’s a fun example of a closed-ended question or a leading question. Closed-ended questions are useful when you want short, simple, informative answers, but they’re conversation killers or thought flow killers. Questions with simplistic and incomplete answers limit the information you can uncover, and you most likely won’t find out more than you anticipate.
For the question “Do you like…” as in “Do you like this feature?” or “Do you like this product concept?” won’t give deep insights, because the answer is generally a yes or no. And then a justification.
If you’re a nerd like me, you’ve probably noticed that when attempting to make small talk and you use the “oh, do you like X?” phrasing, the conversation frequently dies unless you ask a follow-up question. But then if you follow up with another closed-ended question, people start to close up, because it feels like an interrogation not a conversation.
The same is true when you’re asking research questions in qualitative user research. Asking questions back-to-back or asking too many closed-ended questions can feel like an interrogation or like you’re trying to guide a participant's answers. The person answering your questions is probably a little nervous. People have a tendency to be nervous in an interview or research situation, so finding a way to keep your research interviews conversational and structured is key. This means finding the right mix of closed-ended and open-ended questions.
The environment of the research session has a massive impact on the quality and/or quantity of the information gathered. You want to create a space and vibe that is warm and unthreatening, where the participants feel comfortable and at ease. This includes asking questions that assist your participant in making sense of their own thoughts. Our thoughts and words naturally flow more freely and easily when we’re physically comfortable, feel emotionally safe, and have a framework through which to share our thoughts.
Avoid asking questions that will require many rapid follow up questions. Don't create an aggressive environment which often results in limited and unsurprising findings. Create the natural conversational flow that puts a participant at ease. Since the answer to “do you like…” is usually a yes or no, it reveals nothing new. The discoveries from these closed-ended questions are the opposite of what you want.
Since the goal of qualitative research is to discover as much information as possible, your questions are a tool for digging deeper. You want to uncover feelings, thoughts, attitudes, opinions, behaviors that allow you to go on beyond a single instance. You want to see the big picture, find patterns, and fill in the picture with the details of information. This will be a gradual process. But you can help the process along by asking questions that at least allow a free-form answer while supplying support for your participant.
Even asking “what do you like or dislike…” is better. The answer will be more specific, but it requires deliberation and explanation which both create more thought flow. Use the W questions to your advantage and ask questions around the topic like, “why did you try…?” (this can uncover what they need) or “where were you when you tried…?”
In conclusion, asking good qualitative research questions is about helping the participant dive deep. Don’t hinder that process by making it difficult for the participant to share the information they have by asking questions that only require an expected yes or no answer.
