Understanding human behavior is crucial to crafting a survey questionnaire that yields actionable data. After all, when you work with Centiment, real human beings will provide you with their thoughts, opinions, and feedback. As contradictory as it may sound, survey respondents (also referred to as survey panelists and/or survey participants) are often neglected by survey administrators. Or, to specify, their survey experience is neglected which translates to frustrated respondents, poor quality data, and unactionable results.
A good researcher creates survey experiences that keep the respondent engaged without comprising their overall research goal. We’ve outlined best practices in survey design along with question writing mistakes to avoid as you craft your survey. In this article, we’ll go over the methodology that’s made Centiment projects successful over the years: putting our respondents first generates the highest quality of data. We’ll answer the three questions listed below:
***toc
To begin, we’ll define bad behavior in a survey and explain how that can impact your data.
What is bad survey respondent behavior?
It’s easy to tell when a person is behaving badly in public — the reaction of those around them will often make it clear when someone is acting up. In the market research space, defining bad behavior becomes slightly more complicated because each respondent is anonymous, acting alone in a survey build. There are few contextual clues that will indicate when a respondent is behaving badly — and what that means for your survey build as a whole.
There are two key behaviors that indicate a respondent has become disengaged from the survey: straight-lining and poor quality open-text responses. Let’s examine each behavior and where/how it occurs.
Straight-lining is a behavior that we see expressed most often through matrix questions in which respondents select the same response choice for every question in a matrix set — thus creating a straight line of responses down the page. Respondents straight-line through a survey when they’re completely disengaged and are desperate to complete the survey; clearly they want to move on to something more engaging! Once a respondent starts straight-lining, they stop caring — which is bad news for your data set.
Poor quality open-text responses are the second key behavior that results from respondent disengagement. Oftentimes we see respondents behaving poorly in relation to open-text questions when the questions are unclear, confusing, or ask about something the respondent doesn’t know. In that vein, open-text questions need to be used strategically if at all in quantitative market research projects which is why we ask researchers to limit open-text questions to two or fewer per survey experience.
Now that we’ve discussed what we consider to be bad behavior in a survey, let’s explore what causes respondents to act out.
What causes respondents to behave erratically in a survey?
When researchers craft thoughtful, structured surveys, the majority of respondents will behave accordingly: they’ll complete the question set, provide meaningful data, and leave the survey feeling like they contributed to something larger than themselves. When researchers lose sight of the human aspect of market research, respondents react in kind. In poorly structured surveys, respondents are flippant, dismissive, and disengaged. They’ll straight-line through your matrix sets and write nonsensical information as their open-text responses.
When presented with unclear questions or questions requiring an essay in response, survey panelists will disengage and use less than optimal strategies to move past the frustration point. It’s worth repeating that your survey respondents are human beings. They take our surveys in their downtime, on the way home from work or once the kids are in bed.
As such, respondents likely will not have the mental stamina to produce a 15-word response to a survey question -- and asking respondents to push beyond their natural limits for a nominal monetary value frustrates them, which in turn fuels bad behavior. Asking more than two open-text questions per survey experience will quickly disengage even the best-behaved respondents, yielding inconsistent and more difficult to analyze data. Equally, writing a question set with little variety in question format will become a boring experience for respondents which causes disengagement and frustration as well.
So, what can you do to ensure that respondents are engaged so that they don’t behave erratically in your survey?
What steps can researchers take to improve respondent behavior?
The answer is deceptively simple: be thoughtful.
While an academic researcher may have the stamina to answer 30 questions in a single matrix set, the average respondent will not. Put yourself in the respondents’ shoes and remember that they’re here to provide you with feedback — so make it as easy as possible for them to do so! Follow best practice tips to keep respondents engaged by using a variety of question formats, structuring your survey thoughtfully by question type, and avoiding common question-writing mistakes.
Remember, survey questions should be clear and unbiased to every member of your target audience. If you write each question from that perspective, you’ll reduce the opportunities for respondents to behave poorly in your survey. Use text-formatting to your advantage: underline or embolden important phrases that you want respondents to focus on for more complex questions. Define your terms and acronyms (and use the latter sparingly).
Wrap Up
Given that our survey panelists are human beings, there’ll occasionally be a few characters who act erratically even with the best survey design in a large data set; but when a significant portion of your survey respondents are straight-lining or providing poor quality open-text responses, we know that the true problem likely lies in the questions themselves.
Being thoughtful in your design, using a mix of question types, and writing clear questions will enable you to collect top-quality data from our survey panel. If you’re feeling uncertain about writing your question set or designing a positive survey experience, we offer survey questionnaire writing through our services team!
***cta
