A common struggle that our researchers face is understanding the full arsenal of tools at their disposal when they sit down to begin their survey project's lifecycle. We're talking about questionnaire writing, of course! In this article, we’ll define the most common question types used in quantitative online surveys and provide examples that will empower you to write your own great survey questions.
Strong survey questions are clear, unbiased, and yield actionable data that will support your overarching research goal. Understanding the various question types and their different functions will enable you to format your survey build with confidence, knowing that you crafted each question with your end goal in mind.
To begin, there are four general question types:
***toc
The general function of each question type is outlined below:
- Qualifying questions act as gatekeepers to your survey.
- Data-collection questions garner actionable results by collecting respondents’ opinions.
- Attention check questions are tools to measure respondent engagement.
- Demographic questions provide data segmentation opportunities post-collection by contextualizing each respondent.
Let’s examine these question types in greater detail. We’ll describe each question type in sequential order as they should ideally be used in a survey build.
Qualifying Questions
Qualifying questions (also known as in-survey screening questions) should be asked first within a survey. To best respect the time of all respondents, use this question type to immediately identify any respondents who don’t qualify as members of your target audience and route them to the disqualification page.
For this article, let’s pretend that we’re targeting IT-purchasing decision-makers in companies with over 500 employees who have exposure to their CEO. Here at Centiment, we can target over 150+ demographic and firmographic data markers, including purchasing authority by department and employee count, but we don’t have the highly historic specific data for ‘exposure to the CEO.’ As such, we’d introduce a qualifying question with programmed skip logic that will route unqualified respondents (those without exposure to the CEO) out of the survey.
See our sample qualifying questions below:
Question: Do you regularly interact with your company’s CEO?
- Yes (Qualifies)
- No (Disqualifies)
OR
Question: Select which statement below best aligns with your current workplace role.
- I interact with my company’s CEO on a weekly basis. (Qualifies)
- I interact with my company’s CEO on a monthly basis. (Qualifies)
- I interact with my company’s CEO on a quarterly basis. (Qualifies)
- I do not interact with my company’s CEO. (Disqualifies)
Many question formats can be used as qualifying questions, though we see yes/no and multiple-choice questions used most often. For an overview of other common question types used in quantitative studies, take a peek at our quick guide.
Qualifying questions can be as simple or as complex as needed to properly qualify respondents. Please note that the wording of questions can impact the qualification rate -- your project manager will let you know if they anticipate that the presentation of your qualifying question or its corresponding response options will add undue friction in the data collection process.
The more challenging the qualifying question, the more niche we consider the target audience. Our Research Pros are available to craft these crucial questions for you, along with the entirety of your survey build, if you need assistance!
The next question type to explore is the data-collection question.
Data-Collection Questions
Data-collection questions will comprise the bulk of your survey. These questions are conduits for translating your target audience’s feedback and opinions into something actionable that supports your research goal. The positing of these questions can make or break the quality of your data. You’ll need to consider exactly what information you need to gather and determine which format to use for each individual question.
No matter what mix of question formats you use for your data-collection questions, keep these three tips in mind:
- Don’t make assumptions about your target audience, their experiences, or their priorities. Just ask! That’s why survey respondents are here: to answer your questions and provide thoughtful feedback which becomes actionable data.
- Provide enough response choices that survey panelists can answer each question honestly and in a manner that reflects their true values/experiences.
- Be clear and direct. Write each question with the least-educated of your target audience in mind. Define your terms, provide examples when helpful, and avoid technical jargon.
The structure and variety of your data-collection questions will have a direct correlation to respondent engagement as they move through your survey. If you need more tips about how to keep respondents engaged, don’t worry! You can find tips and tricks to maximize respondent engagement in this article about best practices in survey design.
Now that you understand what qualifying and data-collection questions are, let’s move onto attention check questions.
Attention Check Questions
An attention check question is an under-utilized tool to measure respondent engagement. Simply put, attention check questions ensure that your target audience is reading each question carefully, which promotes the best quality of data and allows you to feel confident in your data analysis post-collection.
In your preferred question format, ask respondents to verify that they’re reading each question carefully by requiring them to select a response choice specified in the question text itself. Only respondents who are reading each question will pass on to the remaining portion of your survey -- the rest should be screened out and disqualified. Attention check questions are typically formatted as multiple choice or matrix line questions. If you’d like to write your own attention check question, we suggest taking a look at our standard example. We recommend using one attention check question per survey build.
There’s one common mistake that researchers make when writing their own attention check question: don’t force respondents to select a response choice that boxes them into a perspective that they may not organically share. Below is an example of a poor-quality attention check question in a matrix format:
While this attention check question does require the respondent to read the question text carefully, it misses the mark because it asks respondents to uphold a perspective that they may not share. What if respondents don’t love their job? They may decide to choose a different response option as a show of individualism. For this reason, it’s best practice to make your attention check question neutral rather than aligning the required response choice with something that may turn respondents off.
We recommend placing your attention check within the first twenty questions of the survey, preferably ten to fifteen questions into the build. An attention check question should be simple enough that a respondent only needs to read it carefully once in order to follow its instructions and move forward with the survey.
Finally, we’ve arrived at the final question type, one that is often forgotten by researchers who are just starting out: the demographic question.
Demographic Questions
Demographic questions ask respondents to disclose key characteristics about their lifestyle, health, and other personal details. Firmographic questions are a subset of this question type that revolve around business/employment information; for the sake of simplicity, when we refer to ‘demographic’ questions in this article, we’re referring to firmographic questions too! As your panel provider, Centiment uses our respondents’ answers to demographic questions as data markers, which we can directly target for your research project.
You may be asking yourself -- if demographic information is being targeted on the back-end, before respondents are introduced to my survey, why should I include them in my question set?
The answer is simple: demographic questions allow you to find trends in your data based on background information about your respondents. Most skilled researchers start the data analysis process by segmenting their findings based on collected demographic information such as gender, age, household income, etc. Additionally, back-end demographic targeting doesn’t carry through to your data outputs. Any demographic information required to contextualize your survey or its findings needs to be written into your survey build to be present in the final data output. In short, your demographic questions are important -- don’t neglect to add them into your survey!
Because a survey is similar to a conversation and demographic questions can feel personal, we recommend placing demographics questions at the end of the survey build. By the time respondents reach the end of a well-written survey build, they’ll establish a sense of trust with the process and won’t be put off by requests for demographic information. An exception would be a demographic question that acts as a screening question or has a quota (response cap) attached to it. Once quotas are filled, respondents are screened out and we want to be respectful of their time by placing these questions up front.
Wrapping Up
We’ve examined the differences between qualifying questions, data collection questions, attention check questions, and demographic questions.
In an ideal survey build, you’ll begin with qualifying questions to immediately hone in on your target audience. The structure and number of your qualifying questions will correlate to the ease of introducing respondents to your build. From there, you’ll write your data collection questions. These questions will yield the majority of your findings, so write clearly and use a variety of question formats.
Next, insert an attention check question in your first 15-20 questions as a tool to measure respondent engagement. Disqualify any respondents who don’t select the response choice specified in the question text. Finally, don’t forget to maximize your opportunities for data analysis by including demographic questions at the end of your survey.
