Discover effective survey techniques for balancing the distribution of your survey respondents. Familiarize yourself with simple quotas, compound quotas, and directional balancing to optimize feasibility and minimize fielding costs. Learn practical tips for utilizing these balancing methods to achieve your research objectives without compromising feasibility.
Introduction
When creating a survey, it's common for panel companies to estimate feasibility based on the natural fallout among various demographics and firmographics. Natural fallout refers to the pattern of characteristics that emerge naturally among study participants. Natural fallout provides the most cost-effective way to collect targeted responses.
It's worth considering that specific subgroups within your target population could be particularly relevant to your research objective. Even if you're focusing on a large overall population, these subgroups shouldn't be overlooked, as they can offer valuable insights and aid in analyzing your data effectively.
The task of identifying every required group to obtain a representative sample may seem overwhelming at first. However, we are here to assist you with various tools and methods to manage your audience distribution efficiently.
In this article, we'll explore the various types of balancing and quota tools that are available. Our guidance will also aid in reducing any adverse effects on the feasibility of data collection, pricing, and collection timelines.
Managing the diversity of a research audience and avoiding potentially skewed results is crucial. Quotas and directional balancing are practical tools for this purpose. Proper utilization of these tools can facilitate the identification of differences between key groups, prevent imbalanced data sets, and offer segmentation options for a sample.
What is a quota?
Quotas focus on a narrower and more precise subset within a larger group of people.
Apart from complying with the general criteria of the survey's intended audience, respondents also have to fulfill a supplementary qualification to fit a specific quota. This usually involves demographic or firmographic information, like age, ethnicity, or industry. An instance of this is having a 50% quota for males and a 50% quota for females.
Although quotas seem simple in theory, in reality, they can take on various forms and be measured in different ways. Let's explore the different types of quotas and how they relate to each other and the entire sample.
Types of quotas
There are two main categories of quotas: simple quotas and compound quotas.
Simple quota: A predetermined quantity of individuals who satisfy a single requirement for a particular subcategory. E.g., all male respondents can fit into one simple quota.
Compound quota: A predetermined quantity of individuals who satisfy multiple specific requirements for a particular subcategory. E.g., all male respondents who are also college educated.
The main difference between simple quotas and compound quotas is that compound quotas use multiple criteria to identify a subgroup, while simple quotas only use one qualification. Therefore, it is generally recommended to use simple quotas because compound quotas can complicate the criteria and have a greater impact on audience feasibility.
Now that we understand the different types of quotas available, let's look at the different ways quotas can be measured.
Quota measurement structures
There are three types of quota measures: minimum, maximum, and exact. These each set a specific numeric value for the quota and define the context in which it applies to the larger target audience.
Minimum quotas (most common) set a minimum or floor number of respondents that must meet the criteria of the quota group. While there is no limit or ceiling on the number of respondents who meet these criteria, a certain minimum number must be met.
Maximum quotas set a maximum or capped number of respondents that can meet the criteria of the quota group. There is no minimum requirement associated with this type of quota.
Exact quotas set a precise number of respondents that must meet the criteria of the quota group. Given the constraints this can have on fielding, researchers will often loosen an exact quota to allow for directional balancing with a margin of error of +/- 5%.
You can use these quota measures on both simple and compound quotas to effectively control the allocation of respondents in your survey sample.
Quotas drive specificity, which impacts feasibility and cost
To better understand the impact of quotas on feasibility, we will apply them to the following example of a target audience we are purchasing from a panel company.
Target audience: 1,000 responses from Human Resources (HR) professionals in the US
After reviewing our project goals further, we decide that feedback from HR professionals in the manufacturing industry is a vital component of our research. We set a minimum quota of 30% (300 responses) of the total sample to come from the manufacturing industry.
How can we understand the impacts caused by adding this quota? First, we'll need to look at the manufacturing industry's scale as it relates to the overall job market to derive an approximation.
If 10% of the total job market is made up of manufacturing jobs, you would expect roughly 100 responses to come from HR professionals in the manufacturing industry. By setting a quota of 300 responses on HR employees from the manufacturing industry, we are oversampling by tripling the required manufacturing sector HR personnel relative to the expected natural fallout.
By further specifying this HR audience to oversample based on industry, we are tightening the requirements, which can 1) limit overall feasibility and 2) increase our cost per response as a higher reward must be offered to entice a more specific audience to complete the survey.
Let's go a layer deeper by adding a requirement that 50% of the HR professionals in manufacturing be male by employing a compound quota. 1,000 x 30% x 50% = 150 HR professionals working in manufacturing that are male.
Given males are less common in HR, you complicate the feasibility further by continuing to deviate from the expected natural fallout.
While the sample described above is not uncommon for Centiment Audience Panel to fulfill, you should know the impacts of extended quota requirements. When added to a target audience, the specificity of a quota can significantly impact fielding feasibility and price per response if its requirements vary considerably from the expected natural distribution of the target audience.
Too many quotas create fielding friction
Quotas operate similarly to a target audience. They require specific criteria be met, establish a desired number of respondents who meet those criteria, and have their own feasibility estimates and impacts.
Once quotas fill up or reach their cap, they can obstruct the survey and limit the number of available completion paths. As a result, this can significantly alter the incidence rate. Thus, it is advisable to refrain from implementing too many quotas, especially for a very specific audience.
Compound quotas can also be valuable tools, but as demonstrated in our example, they can drastically complicate fielding by adding complexity to the criteria of a quota.
Similar to over-specifying the requirements of any single quota covered above, using too many quotas, in general, can negatively impact feasibility and price per response.
How and when to use quotas effectively
While quotas can and do impact feasibility, they can also be extremely useful in ensuring a representative sample for key groups. To effectively utilize quotas, we recommend asking yourself the following questions:
- Which subgroups are most important? Do these groups require a minimum viable sample to make the data useful for your research? Ask yourself whether they would still be useful with a natural distribution.
- Can you combine response options to limit the quota count? Take the example of household income, where you have nine response options. Instead of setting nine separate simple quotas, combine the options into three quotas of low, medium, and high income.
- What is the minimum sample necessary for your quotas? Our margin of error calculator can assist you in determining the number of responses required to attain a statistically significant sample.
- How do the quotas look altogether? When utilizing several quotas, assessing them as a whole rather than individually is important. Check for any overlap in the total percentages and whether any quotas can be adjusted to provide more flexibility and improve feasibility.
Using quotas can be a helpful strategy for conducting research that requires segmentation analysis. Although quotas may seem intimidating, keeping these factors in mind can make them a simple and efficient method for obtaining comprehensive data without compromising the feasibility of the target audience.
Directional balancing
Up to this point, we've focused on quotas as a way to manage audience subgroups. Directional balancing is a more relaxed approach to balancing that doesn't require as much specificity.
Directional balancing is the use of quotas written to fall within a specified easement.
Example: 50% female, +/- 5%; in this scenario, the sample can come in between 45% and 55% female and still fall within the required range.
Benefits of directional balancing
Suppose you are looking for general distribution in your sample. In that case, directional balancing can be a great tool to achieve a representative sample while avoiding the tedious work of micromanaging a large set of quotas.
Directional balancing allows greater flexibility by targeting a percentage range vs. a specific minimum, maximum, or exact value. This lessens the impact on audience feasibility by allowing more respondents from the target audience to qualify and, in the case of panel collection, provides a more cost-effective approach for a representative audience.
Conclusion
When it comes to managing subgroups and representation in a target audience, utilizing quotas and directional balancing can be essential methods for obtaining the most valuable research data.
As you consider the best approaches for reaching your audience, keep in mind the different approaches available, and avoid potential pitfalls by asking yourself the following:
- Will natural fallout allow for sufficient representation from my key subgroups?
- Is my target audience broad enough to support multiple quotas?
- Am I focusing my quotas on only the most important handful of attributes?
Ready to put your new quota knowledge to work? Start building for free in the Centiment Survey Tool, which supports every type of quota covered in this article. For further guidance, please reference our Managing Quotas support article.
