Retour

Sampling Error: Definition, Formula and How to Reduce It

Matthieu Saussaye

Sampling error is the difference between a result from a sample and the true value you would get if you surveyed the entire population. It happens because you only ask some people, and a different random sample would give a slightly different answer. It is not a mistake. It is predictable, it shrinks as the sample grows, and for random samples you can calculate it.

The essentials:

  • Cause: chance variation in who ended up in the sample, nothing more.

  • Measured by: the standard error, and reported as a margin of error with a confidence level (for example plus or minus 3 points at 95%).

  • Driven by: sample size (bigger is smaller error), the variability of the answer, and the sampling design.

  • Not the same as non-sampling error: bad frames, nonresponse, leading questions and data errors are separate problems that a bigger sample does not fix.

  • Only calculable for probability samples. For opt-in panels, a margin of error is an approximation, not a guarantee.

This guide goes deeper on the mechanics than our explainer on the survey margin of error, which covers how to read and report the number.

Sampling error vs non-sampling error

Many articles list "selection error", "sample frame error" or "non-response error" as types of sampling error. In the standard survey methodology vocabulary, those are non-sampling errors: they come from how the sample was built or how people answered, not from the luck of the draw. The distinction matters because the fixes are completely different.

Error

Type

Where it comes from

Does a bigger sample fix it?

How to reduce it

Random sampling error

Sampling

Chance: which units happened to be drawn

Yes, it shrinks with the square root of n

Larger n, stratification

Coverage / frame error

Non-sampling

The list or panel misses part of the population, or includes people outside it

No

Better frame, multiple sources, weighting

Selection bias

Non-sampling

Non-random selection, such as self-selection or convenience

No, it can make you more confidently wrong

Random selection, quotas plus weighting

Nonresponse error

Non-sampling

People who answer differ from those who do not

No

Reminders, shorter surveys, incentives, nonresponse weighting

Population specification error

Non-sampling

You defined the wrong population (users vs buyers)

No

Define the population from the decision

Measurement error

Non-sampling

Question wording, order, scales, interviewer effects

No

Pretesting, neutral wording

Processing error

Non-sampling

Coding, data entry, weighting mistakes

No

Checks, double coding, scripted pipelines

Together these make up what methodologists call total survey error. Sampling error is usually the smallest and best understood part of it. For the others, see our guide to survey bias.

The sampling error formula

For a proportion from a simple random sample, the standard error is:

SE = √( p × (1 − p) / n )

where p is the observed proportion (0.40 for 40%) and n is the number of respondents. The margin of error at 95% confidence is about 1.96 × SE. This is the standard result described in the Wikipedia article on margin of error.

Because p × (1 − p) peaks at p = 0.5, pollsters often quote the maximum margin of error, calculated at 50%. Results nearer 10% or 90% have a smaller margin.

Worked example

You survey 400 randomly selected customers; 40% say they would buy a new add-on.

  1. p × (1 − p) = 0.40 × 0.60 = 0.24

  2. 0.24 / 400 = 0.0006

  3. √0.0006 ≈ 0.0245, so SE ≈ 2.4 points

  4. 1.96 × 2.45 ≈ 4.8 points

Report it as "40%, plus or minus about 5 points at 95% confidence": the true share is plausibly between 35% and 45%.

How sample size changes sampling error

Sampling error falls with the square root of n, so halving it takes four times the sample. The table shows the 95% margin of error for a simple random sample (calculated with the formula above).

Sample size (n)

Max margin at p = 50%

Margin at p = 10% or 90%

What it is good for

100

±9.8 pts

±5.9 pts

Directional reads only

200

±6.9 pts

±4.2 pts

Large differences between two groups

400

±4.9 pts

±2.9 pts

Common minimum for a total sample

1,000

±3.1 pts

±1.9 pts

Standard national poll

2,500

±2.0 pts

±1.2 pts

Reliable subgroups of several hundred

10,000

±1.0 pts

±0.6 pts

Precise tracking of small changes

The ±3.1 point figure for 1,000 respondents matches the value given in the Wikipedia margin of error article. To run your own numbers, use our sample size calculator.

Subgroups have their own, larger error

If 1,000 respondents include 150 aged 18-24, any figure for that age group has the error of a sample of 150, about ±8 points at 50%. Plan sample sizes around the smallest group you will report on, not around the total.

Differences have larger error than single results

Comparing two numbers compounds the uncertainty. Pew Research Center explains that in a poll where each candidate's share has a 3-point margin, the margin for the gap between them is about 6 points, so a lead needs to be around 6 points or more before it is clearly larger than sampling error. The same applies when you compare this quarter with last quarter, or segment A with segment B.

Design effects: when your sample behaves like a smaller one

The formula assumes simple random sampling. Real designs change the error:

  • Stratified sampling usually reduces error, because each stratum is guaranteed its share.

  • Cluster sampling usually increases it, because people in the same cluster (school, store, district) answer alike.

  • Weighting increases it. The more unequal the weights, the smaller your effective sample.

The ratio of the actual variance to the simple-random variance is the design effect. With a design effect of 1.5, 1,000 interviews carry about the information of 667 random ones, and the maximum margin rises from about 3.1 to about 3.8 points. Our guide to sampling methods explains each design.

Finite population correction

When you sample a large share of a small population, error is lower than the formula says. Multiply the SE by √((N − n)/(N − 1)), where N is the population size. Surveying 1,000 of a 2,000-person workforce brings the maximum margin from about 3.1 to about 2.2 points. For populations many times larger than the sample, the correction is negligible.

Sampling error in opt-in online panels

Most commercial surveys today use non-probability samples: online panels filled to quotas. Selection is not random, so the textbook formula does not strictly apply. Practitioners handle this in three ways:

  • Report the result without a margin of error and describe the sample ("1,000 panel respondents, quotas on age, gender and region").

  • Report a "credibility interval" or modelled error, stating the assumptions.

  • Use the simple-random formula as a rough guide to precision, explicitly labelled as such.

Whichever you choose, the larger risk in panel research is usually not sampling error but selection and nonresponse error, which no margin of error captures.

Three scenarios: is the difference real?

Scenario

Result

Sampling error check

Verdict

Satisfaction tracker, n = 400 each quarter

72% satisfied in Q2, 76% in Q3

Each figure is about ±4.4 pts; a 4-point move is well within the error of the difference

Not a demonstrated change; keep watching

Concept test, n = 1,000 split evenly between two concepts

Concept A 41% top-2-box, Concept B 33%

Each cell of 500 is about ±4.3 pts at most; an 8-point gap is larger than the roughly 6-point error of the difference

Likely a real preference

Regional cut, n = 1,000 total, 80 in one region

Region X 30% aware vs 45% overall

A base of 80 carries roughly ±10 pts

Treat as a hypothesis, boost the region next wave

The point is not to run formal tests on every cell, but to stop reacting to movements that a re-run of the same survey would plausibly erase.

How to reduce sampling error

  1. Increase the sample size, remembering the square-root rule: going from 400 to 1,600 halves the error.

  2. Stratify on variables related to the outcome (region, customer tier) so each group is represented in proportion.

  3. Boost small subgroups you must report on, then weight them back for totals.

  4. Avoid heavy clustering or account for it in the analysis.

  5. Keep weights moderate. Extreme weights inflate variance; trim them if needed.

  6. Ask more precise questions. A numeric or well-anchored scale has lower variance than a vague one, which shrinks the error of means.

How to report sampling error honestly

  • State n for every figure, especially subgroups.

  • Give the margin and confidence level: "±3 points at 95% confidence".

  • Flag differences that are within the error as "not statistically different".

  • Say which sample type you used, and do not present a probability-style margin for an opt-in sample without saying so.

  • Mention the main non-sampling risks (coverage, nonresponse) in one line. Readers assume the margin covers everything; it does not.

More precision per respondent

Sampling error is about how many people you ask. The other half of a good estimate is how much each answer tells you. SmartInterview runs surveys by voice or text, in each respondent's language, with an AI that follows up on vague answers, and codes open responses into themes automatically. Built-in quota management keeps cells filling to plan, so subgroup bases do not come in short.

You still need the right n. You also get a reason behind each number. Start free.

Frequently Asked Questions

What is sampling error in simple terms?

Sampling error is the natural difference between what your sample says and what the whole population would say. It exists because you asked some people rather than everyone, and it gets smaller as the sample gets larger.

What is an example of sampling error?

If 52% of all your customers prefer option A, a random sample of 400 might show 49% or 55% simply by chance. That gap between the sample result and the true 52% is sampling error.

What is the difference between sampling and non-sampling error?

Sampling error comes from chance in who is selected and shrinks with sample size. Non-sampling error comes from everything else, such as a poor sample frame, nonresponse, biased questions or data mistakes, and does not shrink when you add respondents.

How do you calculate sampling error?

For a proportion from a simple random sample, the standard error is the square root of p(1 - p)/n. Multiply it by 1.96 to get the 95% margin of error. With 1,000 respondents and p = 50%, that gives about plus or minus 3.1 points.

How can sampling error be reduced?

Mainly by increasing the sample size, since error falls with its square root. Stratifying the sample, boosting small subgroups and avoiding heavy clustering or extreme weights also reduce it.

Can sampling error be eliminated?

Only by surveying the entire population, which is a census rather than a sample. In any sample-based study it can be reduced and quantified, but not removed.

Is sampling error the same as margin of error?

Not exactly. Sampling error is the actual gap between a sample result and the true value, which you never observe. The margin of error is an estimate of how large that gap is likely to be at a given confidence level.

Table des matières

Articles associés

Inscrivez-vous gratuitement

Inscrivez-vous gratuitement

Inscrivez-vous gratuitement