Back
Dichotomous Questions: Yes/No Survey Design Done Right
Matthieu Saussaye
A dichotomous question offers exactly two answer options. Yes or no, true or false, agree or disagree, have or have not. It is the simplest closed question there is, and it is the right choice only when the underlying reality is genuinely binary.
The rules that matter:
Use it for facts and events. Did something happen, does someone qualify, do they own the thing. These have two real states.
It is the workhorse of screening. Qualification and routing logic run on yes/no because branching needs a clean split.
Do not use it for attitudes. Satisfaction, agreement, likelihood and importance are spectrums. Forcing them into two boxes throws away everything except the sign.
Yes/no hides intensity. "Would you recommend us?" answered yes covers both the enthusiast and the person who would recommend it if pressed.
Add "don't know" or "not applicable" when they are true states. Without them, uncertain respondents guess, and guesses look identical to answers in your export.
Both options must be equally acceptable to admit. If one answer is socially awkward, the split you measure is partly a measure of embarrassment.
When a dichotomous question fits, and when it does not
The test is not whether a yes/no answer is possible. It is whether the underlying thing you are measuring has two states or many.
What you are measuring | Underlying structure | Right format | What a binary would cost you |
|---|---|---|---|
Did an event occur | Genuinely binary | Dichotomous | Nothing, this is the correct use |
Eligibility for the survey | Binary by definition | Dichotomous | Nothing, but wording must be unambiguous |
Satisfaction or agreement | A spectrum | Rating scale | All intensity, and any movement over time within a side |
Which option they prefer | Several categories | Multiple choice | Every option you did not name |
Frequency of a behavior | A count | Frequency scale or numeric entry | The difference between once ever and daily |
Priority across items | An ordering | Ranking | The trade-off you were trying to observe |
Rows three to six are where dichotomous questions get misused. The formats that belong there are covered in the guides to Likert scales, multiple choice questions and rank order questions.
The legitimate uses
Screening and eligibility
This is the strongest case for a binary. A screener asks whether the respondent belongs in the study, and belonging is a yes or no state. "Have you bought a car in the past 12 months?" either qualifies someone or it does not. Screeners feed routing logic, and routing logic wants a clean split rather than a scale to threshold.
Write screeners so the boundary is unmistakable. "Recently" is not a boundary. "In the past 12 months" is. Ambiguity in a screener does not just add noise, it puts the wrong people in your sample and every downstream number inherits the error.
Behavior that genuinely happened or did not
"Did you contact support about this order?" has two real states. So does "Have you installed the mobile app?" These are facts about the world, checkable in principle, and a respondent either did the thing or did not.
Where behavior questions go wrong is recall. A binary about something six months ago measures memory as much as behavior. Keep the window short enough that the answer is retrievable.
Consent, permission and confirmation
"Do you agree to be contacted about this research?" must be binary. There is no middle state that a legal or operational process can act on. The same holds for opt-ins and confirmations.
Filters that gate a follow-up
A binary is often the cheapest way to avoid asking an irrelevant question. "Did you use the reporting feature this month?" costs one tap and prevents the whole reporting battery being shown to people who have never opened it. Sending everyone through every question is a bigger contributor to fatigue and falling response rates than the question count itself.
The trap: forcing a binary onto a spectrum
Most bad dichotomous questions share one property. They take something continuous and cut it in half, then present the cut as if it were a natural boundary.
"Are you satisfied with our service? Yes / No" is the classic. Satisfaction runs from furious to delighted. Splitting it at an unstated midpoint creates three problems at once:
The cut point is invisible and personal. Each respondent draws their own line between yes and no. You cannot know where. Two people with identical experiences can answer differently.
You lose all movement within a side. If your service improves and merely-satisfied customers become delighted, a yes/no question shows no change at all. Everyone was already a yes.
The result is unstable at the boundary. When a large share of the sample sits near their own personal threshold, small changes in mood or wording flip them, and your headline number moves for reasons that have nothing to do with the business.
The fix is to measure the spectrum and cut it later if you need a binary for reporting. A 5-point scale can always be collapsed into satisfied versus not for a headline slide, and you keep the underlying distribution. A binary can never be expanded back into a scale. Collect the finer measurement; simplify at the reporting stage, and state your cut point when you do.
Why yes/no hides intensity
Consider two respondents who both answer yes to "Would you recommend us to a colleague?" One tells three people a week unprompted. The other would say something positive if directly asked, and never brings it up. In your data they are the same row.
This matters because intensity, not direction, usually predicts behavior. A weak yes and a strong yes have different future value, and the mix between them is exactly what shifts when a product improves or degrades. Scale-based measures such as a 0 to 10 recommendation question exist precisely to separate them.
A binary also gives you a very blunt instrument for spotting change. Aggregate movement in a yes/no split only appears when people actually cross the line. Everything short of that is invisible.
Dichotomous question examples
Fourteen examples, grouped by job. The screening set includes the routing logic, since that is where the format earns its keep.
Screening and eligibility
Have you purchased [category] in the past 6 months? Yes / No. No terminates.
Are you personally involved in choosing software for your team? Yes / No. No terminates.
Do you or does anyone in your household work in market research or advertising? Yes / No. Yes terminates, standard industry exclusion.
Are you 18 or older? Yes / No. No terminates.
Have you used the product at least once in the past 30 days? Yes / No. Yes routes to the active-user block, No routes to the lapsed-user block.
Behavior and fact
Did you contact customer support about this order? Yes / No / I do not remember
Have you installed the mobile app? Yes / No
Did you receive your order within the promised delivery window? Yes / No / Not sure
Have you ever used the export function? Yes / No
Consent and confirmation
Do you agree to take part in this research? Yes / No
May we contact you about your answers? Yes / No
True or false knowledge checks
Before today, had you heard of [brand]? Yes / No
To the best of your knowledge, does your current plan include priority support? Yes / No / I do not know
Is the following statement true of your company: we have a dedicated research team. True / False
Screening logic that does not leak
Screeners are where a badly worded binary does the most damage, because errors there propagate through every result. A few practical rules:
Hide the qualifying answer. If it is obvious that yes gets you into a paid study, some respondents will answer yes. Embedding the real criterion in a list of categories is harder to game than a single loaded yes/no.
Give an explicit time window. "In the last 3 months", not "recently".
Screen one thing per question. "Do you own a car and drive it weekly?" cannot be answered accurately by a weekly driver of someone else's car.
Put the hardest criterion first. Terminate early and you waste less of the respondent's time and less of your incentive budget.
Log the terminations. Incidence per screener question tells you whether the audience exists at the size you assumed, before you commit to a full field.
When to add "don't know" or "not applicable"
A strict yes/no assumes every respondent knows the answer and that the question applies to them. Often neither is true.
Add a third option when:
Genuine ignorance is likely. Questions about plan details, company policy or something that happened a while ago. "I don't know" is real information, and it can be the finding: if 40% cannot say whether their plan includes a feature, that is a communication problem worth knowing about.
The question does not apply to everyone. "Was the delivery driver polite?" has no valid answer for someone who used click-and-collect. Without "not applicable", they pick one at random.
Recall is being tested. "I do not remember" is more honest than a coin flip, and it keeps guesses out of your yes count.
Leave the third option out when the answer is definitionally knowable by the respondent, such as consent, age or whether they personally did something a moment ago. Adding an escape hatch there just gives satisficers a low-effort exit.
One accounting rule matters. "Don't know" is not a midpoint. It sits outside the yes/no dimension entirely, so exclude it from the base when reporting the split, and report its size separately. Folding it into "no" silently converts uncertainty into disagreement.
Analyzing dichotomous data
The data is a binary variable, which makes the arithmetic easy and the interpretation the hard part.
Report a proportion, and say what the base is. "62% yes" needs to state whether that is of everyone asked or of everyone who gave a definite answer.
Use tests built for proportions. A chi-square test of independence or a two-proportion z-test compares yes rates between groups. Small cell counts need care, and Fisher's exact test is the usual fallback.
Model with logistic regression. If you want to know which factors predict a yes, logistic regression is the standard tool for a binary outcome.
Watch the confidence interval. Proportions from small samples are wide. A 60/40 split on 80 respondents is not a reliable majority.
The one-question summary
Ask a dichotomous question when reality has two states. Ask a scale when it has many and you want to know where on the range someone sits. Ask a multiple choice when it has several named categories. Collapsing a scale into a binary at reporting time is always available; recovering a scale from a binary never is.
For the full set of formats and how they fit together, see the survey questions guide, and the companion piece on closed ended questions for the rest of the family.
Two options in, more than two options out
A yes/no answer closes the question. In a real conversation it opens one. SmartInterview runs surveys where a binary can be followed instantly by "what happened?" in the respondent's own voice, so a screening question or a filter becomes the start of an explanation rather than the end of one.
Routing, quotas and terminations work the way you would expect from a survey platform. The difference is that the qualitative half arrives already transcribed and coded into themes, across languages, ready to sit next to your yes/no splits.
See what your screeners have been hiding: start free or get in touch.
Frequently Asked Questions
What is a dichotomous question?
A dichotomous question is a closed question with exactly two answer options, such as yes/no, true/false or agree/disagree. It is the simplest form of closed question and is best used when the thing being measured genuinely has two states, such as whether an event happened or whether someone qualifies for a study.
What is a dichotomous questionnaire example?
A survey screener is the standard example: "Are you 18 or older?", "Have you purchased in the past 6 months?", "Are you personally involved in choosing software for your team?", each answered yes or no, with a no on any of them terminating the interview. The format suits screening because routing logic needs a clean, unambiguous split.
When should you not use a dichotomous question?
When the underlying reality is a spectrum. Satisfaction, agreement, likelihood, importance and frequency all have many levels, and a yes/no forces each respondent to apply their own invisible cut point. You lose intensity, you lose any movement that does not cross that line, and the result becomes unstable for people sitting near their own threshold.
Should a yes/no question include a "don't know" option?
Include one when genuine uncertainty is likely, such as questions about plan details, company policy or older events. Without it, unsure respondents guess and their guesses become indistinguishable from real answers. Leave it out for things the respondent definitionally knows, such as consent or their own age. Report "don't know" separately rather than folding it into "no".
What is the difference between a dichotomous question and a multiple choice question?
A dichotomous question has exactly two options; a multiple choice question has three or more and may allow several selections. Technically a dichotomous question is the two-option case of multiple choice, but it behaves differently in practice: it is used for facts and routing rather than for measuring which of several categories applies.
How do you analyze yes/no survey data?
Report the proportion answering yes and state the base clearly. To compare groups, use tests designed for proportions such as a chi-square test of independence or a two-proportion z-test, with Fisher's exact test when cell counts are small. To model which factors predict a yes, use logistic regression. Always check the confidence interval, since proportions from small samples are wide.
