No matter the number of survey respondents, generating the actual views of every member of the represented population is impossible (unless you survey every member of a population!). This is why, for every survey you conduct, you need to consider the margin of error (MOE).
Let's dive deeper to understand the margin of error and how to handle it. Get ready to crunch some numbers.
Free template to analyze your survey results
Analyze your survey results in a way that's easy to digest for your clients, colleagues or users.
Use template
What is the margin of error in a survey?
The margin of error is the range within which the actual value of a survey parameter falls within a certain confidence level. In other words, it quantifies how the results from a sample might differ from the actual value you would have obtained if you had studied the whole population.
The confidence interval (CI) is the range between the upper and lower bounds of an estimate. The narrower the interval, the better and more accurate your results.
For instance, suppose a random sample of people has a +/−2% margin of error at a 95% confidence level. That implies that if you repeated the same survey 100 times, you expect the percentage of people who gave a particular answer to fall within the range of 2% of the reported results, 95 times.
The margin of error usually decreases with an increase in the random sample size or the number of responses. That means you can be more confident your results are reliable.
When can you use a margin of error?
You use a margin of error when you have a random sample—a set of randomly selected respondents from the whole population you’re studying. A random sample is also known as a probability sample since every member of your population has a known probability of being part of your sample.
For example, your company wants to know whether workers would prefer an extra leave day or bonus pay. You can randomly select a few employees from your workforce and ask them to choose their preferences. A margin of error will help the decision-makers understand the accuracy of the results.
What factors affect the margin of error?
The size of the margin of error depends on various factors:
Sample size: the more respondents who complete your study, the smaller the margin of error
Confidence level (CL): increasing the confidence level leads to a wider margin of error
Population variance: the higher the variance, the larger the margin of error
Poll design: your questions' exact wording can influence how people answer them, affecting the MOE
Response rate: a lower response rate increases the MOE. It can also increase if the respondents don't closely resemble the larger population.
Non-sampling errors: errors from sources such as coding and measurement can affect the margin of error
What is an acceptable margin of error for a quantitative survey?
The MOE size varies with the percentage of the target population sampled. Knowing the MOE for a particular study is crucial since the value for all percentages will be assessed through this lens when reviewing data.
If you were surveying 1,000 people using a confidence level of 90, surveying at least 250 people would give you an MOE of 4%. With this margin, you could be reasonably confident that the results were reflective of the audience you were surveying.
How to calculate the margin of error
Whenever you perform a statistical survey, you should calculate the MOE. Before you do so, you need to define your population.
A population is a group of elements from which you want to survey and gather data. Properly defining the population and selecting an appropriate sample size can help reduce the MOE.
Margin of error calculator
Optimize your research’s impact when you improve the margin of error.
0%
Margin of error
The total number of people whose opinion or behavior your sample will represent.
The probability that your sample accurately reflects the attitudes of your population. The industry standard is 95%.
The number of people who took your survey.
0%
Margin of error
Here's the formula:
MOE= z-value √ (p(1-p)/n)
Where:
n is the sample size
p is the sample percentage
z-value is the critical value corresponding to your confidence level
Here are standard confidence levels with their corresponding z-values:
90%: 1.64
95%: 1.96
99%: 2.58
Finding the maximum margin of error formula
For the maximum MOE, you'll use 0.5 as p and the z-value for 95% (the standard confidence level), which is 1.96. Thus, the formula is a direct transformation of the sample size:
Maximum 95% MOE = 1.96√0.5 (1 - 0.5)/n =0.98/√n
Plug in your sample size to the formula, and voila! You'll get the maximum margin of error for whatever survey you've done. That means the figure is unique for a given size.
For example, the maximum MOE for a sample size of 100 would be 0.98/√100 = 0.098
How to calculate the margin of error with your survey data
Now you have your formula, crunching those figures won't be a headache.
Take n, the sample size, and p, the sample percentage.
Calculate p(1 - p) and divide the result by n.
Find the square root of the value.
Multiply the figure you got in Step 3 by the z-value.
An example of calculating the margin of error
Suppose you surveyed 1,000 respondents to find out their views on volunteering. 500 of them agreed that volunteering is an excellent part of life. What's the margin of error given a 50% CL?
MOE= z√(p (1 - p))/n
P = 500/1,000 = 50%
z-score = 1.96 (at 95% confidence level)
Therefore, MOE= 1.96√ (0.5(1 - 0.5))/1,000
=0.031=3.1%
So the sample's MOE is +/−3%. That implies if you repeated the survey several times, the number of people who love volunteering would be within 3% of the sample percentage (50%) 95% of the time.
How sample size affects the margin of error
The size of the sample is a key factor that affects your margin of error. Looking to increase your study’s precision? Interview more respondents and ensure they complete your survey.
Increasing sample size reduces the standard deviation (a measure of your estimate's variability). In other words, it narrows the range of possible values, leading to more precise estimates and lower MOE.
For example, if you're looking to estimate the average income of your existing customers, a sample of two people will lead to a wide range of figures, so the margin of error will be high. But if you increase the sample size to 1,000 people, the MOE will be significantly narrower.
That means MOE is critical to understanding whether your sample size is appropriate. Is it too large? You'll need to survey more people to capture your population's attitudes accurately.
Calculate your ideal sample sizeHow to increase your data's reliability
If you're looking to boost your survey's reliability, the trick is to minimize the MOE. Here are three tried-and-true ways you can generate more accurate results:
1. Minimize the variables
A high number of variables can introduce more errors to your survey. Variables cause your standard variation to shoot up, increasing the MOE. Here's a trick: change how you collect data. For example, ensure the process is rigorous and measure your factors or variables accurately.
2. Increase the sample size
This hack is often the easiest on our list. Statistically, if more people complete your study, your chances of getting a representative response increase because the confidence interval decreases. The result is a lower MOE. But ensure you have enough resources and time to generate a larger sample size.
3. Lower your confidence level
Using lower confidence is another trick that can lead to a narrower MOE. But be careful since a lower level means decision-makers will be less confident in the results.
Therefore, only reduce the CL if the cons of a smaller MOE outweigh the cons of a lower CL. For example, if it’s unfeasible to increase the sample size due to costs, you can reduce the CL to achieve a narrower interval.
You can calculate the margin of error using the following formula:
MOE= z √(p (1-p))/n
Where z is the z-value for your desired confidence level, p is the sample percentage, and n is the sample size.
A large MOE implies a high chance of the actual value being very different from the value you've estimated. It often occurs when dealing with small sample sizes and high data variability, combined with a high CL.
If the margin of error is very high, the survey's sample results won't accurately represent your population, so decision-makers can't rely on it. The solution is to survey more people, reduce variables, lower the CL, or use a one-sided CI.
