top of page

No such thing as a "bad respondent," just bad screening.

Patricia Roller


If Tom Hanks would sneak into a Vidlet study, we’d be able to catch him.

It’s easy to become cynical about your study participants when you work in qualitative research. Participants often pretend to be someone they are not. We have seen it all -- changed names, sunglasses, wigs, fake LinkedIn profiles -- all done so participants can sneak into a study that pays little.

Our database has a button for that – “bad respondent”. We keep their video in the database to make sure that we know what they look like.

The industry may have inadvertently caused this problem by requesting that study participants have not done a study within a certain timeframe – a name change is a convenient way to break that rule.

Participants are surprised when we call them out. They tell us that they are often invited into a focus group because they always say the right thing, meaning that they praise a product or experience and nobody cares who they actually are.

A shift in how we view our participants happened when we reached out to a “bad respondent” named “Lisa.” She showed us an older, used car when the study required participants to have bought a new car in the past 3 years. When we told her we couldn’t include her in the study, she lamented and told us how desperately she needed the money. She had lost her job due to COVID-19 and had just started to drive for Doordash to make ends meet. We deleted her unusable videos and paid her anyway. A week later, we landed a study to understand drivers in the gig economy. Lisa was the first person that came to mind. She was happy to get the call and we were happy that we knew about her new role. Her insights sparked many ideas and gave us a sense of what it is like to be a delivery driver in current times.

This was the moment when we realized that our participants, fake or not, all have something to contribute, if we only know enough about them. Since then, we don’t just brush off wrong participants as “bad”, but we go back and ask more questions.

Another participant showed up as a “home-bound IT professional” but had nothing meaningful to say. When he mentioned field work, we would have normally put him on the “bad respondent” list. But lucky for us and him, we also had a study on field workers. His insights in the study that matched his experiences were rich and full of details on what it is like to be a field worker for AT&T.

This week we are launching a screener form that allows participants to tell us as much as possible about themselves. This will help us find the right people faster and allow our clients to get better results. It took us over a year to build our new database for video-verified participants and the timing couldn’t be better.

We will invite Tom Hanks to a study about acting anytime, he just can’t pretend to be French.

Image Source: YouTube

12 views0 comments


bottom of page