Back

Concept Testing: Ultimate Guide on Methods & Best Practices

Matthieu Saussaye

Start here: SmartInterview

If you want to test a concept this week rather than next quarter, start with SmartInterview. You show each respondent a concept, collect the closed scores you need (purchase intent, uniqueness, believability), and an AI follow-up asks why whenever an answer is thin. Respondents can answer by voice or text, in their own language, and the open answers are coded into themes you can count.

Try SmartInterview free: start with 50 respondents, no credit card, and run your next concept test on it.

Below: the short answer, the four methods compared, a step-by-step process with real question wording, how to read the results, and the mistakes that quietly ruin most concept tests.

The Short Answer

Concept testing is research that shows an idea for a product, service, ad, package or price to people in your target market before you build or launch it, and measures whether they understand it, want it and believe it. It tells you which idea to back, what to fix, and which idea to drop while changing it is still cheap.

  • What you test: a concept board, a short description of the idea with its benefit, its reason to believe, and usually a price and a visual.

  • What you measure: purchase intent, appeal, uniqueness, believability, relevance and value for money, plus the open-ended reasons behind each score.

  • The four main designs: monadic (one concept per person), sequential monadic (several, one after another), comparative (side by side) and protomonadic (one alone, then a comparison).

  • Sample: plan on roughly 100 to 200 completes per concept cell if you want to compare concepts with any confidence. Fewer is fine for directional reads.

  • The rule that matters most: judge a concept against a benchmark (your current product, a past winner or a control concept), never against an absolute number. Stated purchase intent always overstates real buying.

Concept Testing Methods Compared

Most of the choice comes down to one question: do you need to know how good each idea is on its own, or which of several ideas wins? The table shows the trade-offs.

Method

What each respondent sees

Sample needed

Best for

Main risk

Monadic

One concept only

100 to 200 per concept (cells multiply)

Clean, realistic read of each idea; go/no-go decisions

Expensive with many concepts

Sequential monadic

Two to four concepts, one at a time, in rotated order

150 to 300 in total

Screening several ideas on a smaller budget

Order effects and fatigue: the first concept scores differently

Comparative (paired)

Two or more concepts side by side

100 to 200 in total

Picking a winner between close variants

Tells you which is preferred, not whether either is good

Protomonadic

One concept alone, then a comparison with the others

150 to 200 per starting cell

Getting both an absolute score and a preference

Longer survey; the comparison is biased by the first concept seen

Qualitative interviews

One or more concepts, discussed in depth

8 to 20 interviews

Early ideas, understanding reactions and language

No reliable numbers; a few strong voices can dominate

Which method to pick

Your situation

Use

Why

One idea, a go/no-go decision

Monadic against a benchmark

Nothing else in the survey colors the reaction

Three to six early ideas, limited budget

Sequential monadic, rotated

Everyone sees everything, so the sample stays small

Two near-identical variants (name, price, pack)

Comparative

Small differences show up only side by side

Final shortlist before launch

Monadic, one cell per finalist

The read closest to how the market will meet the product

You do not yet know what the idea should be

Qualitative interviews first

You need words and reasons before you need scores

What Is Concept Testing?

A concept is an idea described well enough that someone can react to it as if it already existed. Concept testing puts that description in front of the people who would buy it and measures their reaction in a structured way, so the decision does not rest on whoever argues hardest in the meeting.

It sits between two other kinds of research. Before it, exploratory work (interviews, open questions, see qualitative vs quantitative research) tells you what problem is worth solving. After it, in-market tests and A/B tests tell you what real customers actually do. Concept testing is the cheapest point in that chain at which you can kill or reshape an idea.

What concept testing is not

  • Not a usability test. Usability asks whether people can use the thing. Concept testing asks whether they want it.

  • Not an A/B test. An A/B test measures behavior on something already live. A concept test measures stated reactions to something that does not exist yet.

  • Not a forecast. A concept score is a relative signal. Turning it into a sales volume needs calibration against past launches, which most companies do not have.

What You Can Concept Test

Type

Typical question

What goes on the concept board

New product or service

Is there demand, and from whom?

Need, benefit, how it works, price

Feature or line extension

Does it add value for current customers?

The new feature next to the current offer

Positioning and claims

Which promise is most motivating and believable?

The same product with different headlines and reasons to believe

Price

Which price point keeps appeal without leaving money on the table?

The same concept at different prices, one per cell

Name, packaging, logo

Which execution fits the brand and stands out?

Visuals, shown monadically or side by side

Ad or campaign idea

Is the message understood and does it change intent?

Storyboard, key visual or script

Anatomy of a Good Concept Board

Most bad concept tests are lost before the first respondent arrives, because the concept itself is unclear. A concept board that tests well has the same parts every time:

  1. Insight: the consumer tension in their words. "I want to eat better at lunch but I only have ten minutes."

  2. Benefit: what the product does for them, in one sentence.

  3. Reason to believe: why the benefit is credible: an ingredient, a technology, a proof point.

  4. How it works or what it is: format, size, where to buy, how to use.

  5. Price: if price is part of the decision, show it. A concept without a price tests a fantasy.

  6. Visual: a simple sketch or mockup at the same level of finish for every concept.

Keep every concept at the same level of polish. A glossy render next to a text description is a test of your designer, not of the idea. Keep length similar too: a concept with twice as many benefits will usually win on appeal and lose on believability.

The Metrics That Matter

Use the same core questions for every concept and every test, so results become comparable over time. Five-point scales are the usual standard (see how to build a Likert scale).

Metric

Question wording

How to report it

Purchase intent

"How likely would you be to buy this product if it were available at [price]?" Definitely would / Probably would / Might or might not / Probably would not / Definitely would not

Top-2-box (definitely + probably), and top box alone

Uniqueness

"How new and different is this product compared with what is available today?"

Top-2-box

Believability

"How believable are the claims made about this product?"

Top-2-box

Relevance

"How well does this product meet a need you have?"

Top-2-box

Value for money

"Considering the price, how would you rate the value?"

Top-2-box; watch for big gaps with intent

Likes and dislikes

"What, if anything, do you like about this idea?" / "...dislike?"

Coded themes with counts and quotes

Understanding

"In your own words, what is this product?"

Share who described it correctly

Top box often matters more than top-2-box. "Probably would buy" is a polite answer. Definite intent is the stronger signal of a concept people will actually go and look for.

How to Run a Concept Test, Step by Step

Step 1: Write down the decision

Not the objective, the decision. "Choose one of three positionings for the autumn launch" is a decision. "Understand consumer reactions" is not. Write the action you will take for each possible result before you field, including what result would make you stop.

Step 2: Pick the benchmark

Decide what each concept is compared against: your current product, a concept you launched before, or a deliberately ordinary control concept. Without a benchmark, a 40% top-2-box score means nothing. With one, it means "better or worse than what we already know".

Step 3: Build the concept boards

Use the six parts above, equal polish and equal length for all concepts. Then show each board to three people who match your target and ask them what the product is. If they cannot say it back, rewrite it before you test it.

Step 4: Choose the design and the sample

Use the method table above. For sample, the useful number is the margin of error per concept cell. At 150 respondents, a 50% result carries a margin of about plus or minus 8 points at 95% confidence; at 300 it is about 5.7 points. Two concepts 5 points apart with 150 people each are not different. Work out your own numbers with the sample size calculator.

Who you ask matters more than how many. Screen for the category: people who buy or use it, at the frequency your business depends on. A large sample of the wrong people gives a precise answer to the wrong question.

Step 5: Write the questionnaire

A concept test that takes 8 to 12 minutes keeps attention high. The order matters:

  1. Screener: category use, purchase frequency, quota variables (age, region, language).

  2. Current behavior: what they buy today and why, asked before the concept so it does not color the answers.

  3. Concept exposure: the board, with enough time to read it.

  4. Open reaction first: "What is your first reaction to this idea?" before any scale, so the closed questions do not put words in their mouth.

  5. Core metrics: intent, uniqueness, believability, relevance, value, in the same order for every concept.

  6. Diagnostics: likes, dislikes, understanding, what would make them more likely to buy, what it would replace.

  7. Profile: demographics at the end.

Avoid the classic traps: leading wording, double-barreled questions, and scales that differ between concepts. The survey bias guide covers each one with examples.

Step 6: Field, with probing

The open questions are where you learn what to fix, and they are also where respondents write "good" and "not sure". A follow-up such as "What makes it feel expensive to you?" turns a one-word answer into a reason. In an AI survey the follow-up is generated from what the person just said; in a traditional survey you have to anticipate it. See qualitative research with AI for how that probing works.

Step 7: Analyze and decide

Compare each concept with the benchmark on the core metrics, then read the coded open answers to explain the gaps. Look at the scores for the segments the business actually depends on, not just the total. Then apply the decision you wrote down in step 1.

How to Read Concept Test Results

Pattern

What it usually means

What to do

High intent, low uniqueness

People want it, but it looks like what exists

Strengthen the difference, or compete on price and distribution

High uniqueness, low believability

Interesting, but nobody trusts the promise

Add or sharpen the reason to believe

High appeal, low value for money

The price is the barrier

Test a lower price point or a smaller entry format

Middling everything

The concept is unclear or tries to do too much

Check understanding; cut to one benefit

Strong in one segment, weak overall

A niche product, not a mass one

Size that segment before you decide

Beats the benchmark on top box

A real candidate

Go to the next stage, and keep the open answers for the launch copy

A worked example (hypothetical)

Illustrative numbers, not a real study. A Swiss snack brand tests three concepts for a high-protein lunch bar, monadic, 150 category buyers per cell in German and French, against the brand's current bar as a fourth control cell.

Concept

Purchase intent (top box)

Top-2-box

Uniqueness (T2B)

Believability (T2B)

Current bar (control)

14%

41%

22%

63%

A: "Lunch in 2 minutes"

23%

52%

35%

58%

B: "Swiss alpine protein"

16%

47%

48%

44%

C: "No added sugar"

15%

44%

25%

66%

Concept A is the only one clearly ahead of the control on top-box intent (9 points, just beyond the margin of about 8.7 points for a difference between two cells of 150). Concept B is the most distinctive but has a credibility problem, and the open answers explain it: people asked what "alpine protein" actually is. The decision: develop A, and borrow B's Swiss origin as a reason to believe once it is explained in plain words.

Common Concept Testing Mistakes

  1. Testing without a benchmark. An absolute score has nothing to be compared with. Always include a control.

  2. Reading stated intent as sales. People overstate what they will buy. Use intent to rank concepts, not to forecast volume.

  3. Unequal concept boards. Different polish, length or number of benefits decide the result before respondents do.

  4. Hiding the price. Appeal without price is free. If price will be on the shelf, it belongs on the board.

  5. The wrong sample. General-population respondents rating a category they never buy produce confident noise.

  6. Ignoring the open answers. Scores tell you which concept wins; the verbatims tell you why and what to change. Most teams never code them.

  7. Too many concepts per person. After three or four, respondents tire and later concepts score lower for reasons that have nothing to do with the idea.

  8. Treating one test as the final word. The best teams test, revise and retest. A second round on a sharpened concept is cheap compared with a failed launch.

A Note on the "95% of New Products Fail" Statistic

Many concept testing guides open with the claim that 95% of new products fail, often attributed to Harvard or MIT. The figure is widely repeated and poorly sourced, and published estimates of new product failure rates vary a lot by industry and by how "failure" is defined. You do not need the scary number: if a launch costs more than a few weeks of research, a concept test that kills or fixes one weak idea has already paid for itself.

How AI Changes Concept Testing

The method has not changed in decades. What has changed is the cost of the parts that used to be slow.

  • Probing at scale. An AI follow-up asks every respondent "why?" in their own words, so you get interview-like reasons from a quantitative sample.

  • Voice answers. Respondents who would type three words will often speak three sentences, which is exactly what the diagnostic questions need (see voice surveys vs traditional surveys).

  • Coding in minutes. Open answers are grouped into themes and counted automatically, with each theme linked back to the original quotes.

  • Several languages in one study. In a market like Switzerland, the same concept can be tested in German, French and Italian with one questionnaire.

What AI does not replace: deciding who to ask, writing a clear concept, choosing the benchmark and making the call. Be wary of "synthetic respondents" generated by a model in place of real people: they can help you draft a concept, but they cannot tell you whether real buyers want it.

For a real example, see how Pharmalp tested the positioning and price of new products with SmartInterview. To run your own, you can start a concept test free with 50 respondents.

Frequently Asked Questions

What is concept testing?

Concept testing is market research that shows a product, service, ad or price idea to target customers before launch and measures their reaction: whether they understand it, want it, find it different and believe it. It is used to choose between ideas and to fix weak points while changes are still cheap.

What are the main types of concept testing?

The four standard designs are monadic (each respondent sees one concept), sequential monadic (each sees several, one at a time, in rotated order), comparative (concepts side by side) and protomonadic (one concept alone, then a comparison). Qualitative interviews are often used before them, on early ideas.

How many respondents do you need for a concept test?

For comparing concepts, plan on roughly 100 to 200 completes per concept cell. At 150 per cell, a result near 50% has a margin of error of about plus or minus 8 points at 95% confidence, so smaller differences are not reliable. Directional early tests can use fewer.

What questions should a concept test ask?

The core set is purchase intent, uniqueness, believability, relevance and value for money on five-point scales, plus open questions on first reaction, likes, dislikes and understanding. Ask the open reaction before the scales, and keep the order identical for every concept.

What is the difference between concept testing and product testing?

Concept testing measures reactions to a description of an idea before it exists. Product testing (or a home-use test) puts a working product in people's hands and measures their experience of using it. Concept testing usually comes first.

How long does a concept test take?

A traditional agency study often takes several weeks from brief to report. With one of the best concept testing platforms that has its own panel and automatic coding, fielding and a first read can take a few days, as long as the concept boards and the decision are ready before you start.

Table of contents

Related articles

Sign up for free

Sign up for free

Sign up for free