Back
How to Validate a Product Idea With Real Customers
SmartInterview Team
The Short Answer
To validate a product idea, stop asking people what they think of it and start asking what they already did about the problem. Recruit people who have the problem now, ask about specific past events, and treat any commitment of time, money or a referral as the only real yes.
The core rule: ask about behaviour that has already happened. Opinions about the future are free and unreliable. What someone did last month is a fact.
Two questions to delete: "Would you use this?" and "How much would you pay?" Both invite a friendly guess that predicts nothing.
How many conversations: plan for 15 to 20 within one segment. Patterns typically repeat by then, and if they do not, your segment is too broad.
Then quantify: qualitative work tells you what the options are, a larger survey tells you how common each one is. Neither replaces the other.
Bad Questions, Better Questions, and What They Reveal
Most useless discovery interviews fail on question design, not on interviewing skill. The failing pattern is always the same: the question asks the person to predict their own future behaviour, or to grade your idea, and both are things humans do badly and politely.
Bad question | Why it fails | Better question | What it actually reveals |
|---|---|---|---|
"Would you use a product that does X?" | Asks for a costless prediction, and the polite answer is yes. Intention to use is a weak predictor of use | "Walk me through the last time you dealt with X. What did you do?" | Whether the problem occurs at all, how often, and what the current process really is |
"How much would you pay for this?" | Nobody can price something they have not used. You get an anchor pulled out of the air, usually too low or flattering | "What are you spending on this today, in money, tools or people's time?" | The real budget envelope, and whether a budget line exists at all |
"Do you think this is a good idea?" | Invites them to be a critic rather than a customer, and criticism from a non-buyer is noise | "Who on your team would be affected by this, and who would have to approve it?" | The buying unit, and whether the person you are talking to can decide anything |
"Is X a problem for you?" | Leading. Almost anything sounds like a problem when a stranger names it. Nearly everyone says yes | "What is the most frustrating part of your week?" | Whether your problem shows up unprompted, and where it ranks against everything else |
"Would you pay 50 a month for this?" | Anchors them on your number and turns a discovery call into a negotiation | "Have you paid for anything to solve this before? What happened to it?" | Proven willingness to pay, and why previous solutions were abandoned |
"What features would you want?" | Customers describe solutions in the shape of what they already know, and you inherit their design instead of solving their problem | "If this worked perfectly, what would change about your day?" | The outcome they are buying, which is what your positioning has to speak to |
"Are you interested?" | Interest costs nothing and expires immediately | "Can I put 30 minutes with your ops lead in the calendar next week?" | Whether the interest survives contact with a small cost |
The Mom Test Principle
Rob Fitzpatrick's book The Mom Test gives the idea its name, and the premise is simple: a good question is one your mother could not lie to you about, because it asks for facts about her life rather than for an opinion about your idea. Ask "do you think my app is a good idea" and she will say yes. Ask "how did you book your last three holidays" and you get the truth, because she is just describing what happened.
Three rules follow from it.
Talk about their life, not your idea. The moment you describe the product, the conversation changes register. They stop reporting and start reacting. Keep the pitch out of the room until you have what you came for.
Ask about specifics in the past, not generics about the future. "How do you usually handle X" produces an idealised description of a process nobody actually follows. "Show me what you did the last time" produces the real one, including the spreadsheet they are embarrassed about.
Listen more than you talk. If you are talking for more than about a third of the call you are pitching, not learning. Silence after an answer is the highest-yield technique in the entire process, because people fill it with the qualification they were holding back.
Compliments, fluff and the three things to write down
Compliments ("that's really cool", "I'd definitely use that") feel like progress and contain zero information. Fluff is anything hypothetical, generic or future-tense ("I would normally", "I always try to"). Both should be noted as social noise and ignored when you review your notes.
What matters is concrete: a specific past event with a date attached, money or time already spent, and an emotion attached to a real incident. Those three things are the only entries in your notes that should influence a decision.
Recruiting the Right People to Talk To
Bad recruiting invalidates good interviewing. If you talk to twenty people who do not have the problem, you will get twenty polite, coherent, useless conversations, and you will not be able to tell that anything went wrong.
Define the screen before you look for anyone
Write down, in one line, what someone must have done recently to qualify. Not who they are, what they did. "Has run a paid ad campaign in the last 60 days" is a screen. "Marketing manager at a mid-size company" is a job title, and job titles do not have problems, people do.
Where to find them
Second-degree network. Ask contacts for introductions to people who match the screen, rather than talking to the contacts themselves. One step of distance removes most of the politeness bias.
Where they already complain. Communities, subreddits, industry forums, review sites for the tools they currently use. People who wrote a two-paragraph complaint about a competitor are pre-qualified and usually happy to talk.
Your own funnel. Anyone who signed up and did not activate is a strong interview target, and easier to reach than a stranger.
Paid recruitment or a panel. When you need people you have no route to, especially in a market you are not part of. Screen hard, and expect a proportion of professional respondents. How panels work, and their failure modes, is covered in our guide to qualitative and quantitative methods.
One warning about incentives: paying for time is normal and fine, but if the incentive is large relative to the person's hourly rate, you start attracting people optimising for the incentive rather than people with the problem. Pay enough to respect the time, not enough to be a job.
Structuring a Discovery Interview
Thirty minutes is enough. A workable structure looks like this, and the timings matter because the temptation is always to spend the whole call on the last section.
Frame it honestly (2 minutes). "I'm trying to understand how teams handle X. I'm not selling anything today and there are no right answers." This is true, and it lowers the pressure to be encouraging.
Context (5 minutes). Their role, their week, how the relevant work is structured. You are building enough understanding to ask a sharp follow-up later.
The last occurrence (10 minutes). The heart of it. Pick the most recent time the problem occurred and walk through it step by step. Who was involved, what tools were open, how long it took, what went wrong, what they did afterwards.
Prior attempts (5 minutes). What they have tried, bought, built or evaluated. What happened to each. Why they stopped. This is where you find out whether there is budget and what the real objection to a new tool will be.
Magnitude (3 minutes). How often, how much time, what it prevents. You are turning a story into something you can size.
The commitment ask (5 minutes). Only now, if at all, describe what you are building, and ask for something with a cost: an introduction to a colleague, a follow-up with the person who owns the budget, a letter of intent, a pre-order, a slot in a pilot. Their reaction to a small cost is worth more than everything they said before it.
How many conversations before patterns emerge
Within a single, tightly defined segment, most teams find the same themes repeating by the twelfth to twentieth conversation. The useful stopping rule is not a count, it is saturation: when two consecutive interviews teach you nothing you have not already heard, you have enough for that segment.
Two failure modes distort this. If you are still surprised at conversation 25, you are almost certainly talking to several different segments at once and should split them and treat each separately. If nothing is new by conversation 6, you may have recruited people who are too similar, often because they all came from the same source.
Segments do not pool. Fifteen conversations with agencies and five with in-house teams is not twenty conversations, it is a finished study and a broken one.
Doing this at more than interview scale
Manual interviews stop being practical somewhere past a couple of dozen, and that is usually the moment you need breadth: several segments, several markets, or a check on whether a pattern you found in one country holds in another. This is where an AI-moderated conversation is genuinely useful, because it can ask the same open question of several hundred people and probe each answer with a follow-up rather than accepting the first sentence.
That is the job SmartInterview was built for: open questions with automatic follow-up probing, in the respondent's own language, with the open responses coded afterwards so you can see how often each theme appears rather than reading 400 transcripts. It does not replace the first twenty conversations. Those you should still run yourself, because the point of them is to change your mind.
From Qualitative Signal to a Quantitative Test
Interviews tell you which hypotheses are worth testing. They cannot tell you how common anything is, and treating "four out of twelve people said" as a statistic is one of the more expensive mistakes in early product work.
The handoff has a shape.
Turn what you heard into closed questions. The answer options should be the actual workarounds people described, in their words, not categories you invented. If your options do not include what people really do, everyone picks "other" and you learn nothing.
Measure incidence first. What share of the target population has the problem, at what frequency, and what they currently use. This tells you if the market is big enough before you worry about the product.
Test the message, not the idea. A landing page with a clear promise and a real call to action measures response to positioning. Keep the traffic source constant between variants or you are measuring the channel.
Then test willingness to pay with a real transaction. A pre-order, a deposit, a paid pilot. Stated price sensitivity and actual conversion at a price are different quantities, and only one of them is a fact.
Be careful with the smoke test. A landing page conversion tells you the promise is attractive. It does not tell you the product can deliver it, and a strong signup rate on a promise you cannot keep is an expensive false positive.
Avoiding Confirmation Bias When It Is Your Idea
You are the worst possible person to evaluate your own idea, and you are also the person who has to. Some structural defences work better than resolving to be objective.
Write the kill criteria first. Before the interviews, write down what result would make you stop. "If fewer than half have tried to solve this in the past six months, the problem is not urgent enough." Deciding in advance is much easier than deciding once you have the data you did not want.
Separate notes from interpretation. Record what was said in the person's words, in one document. Put your conclusions in a different one. Interpretation contaminates memory fast, and once you have written "she loved it" you will never recover what she actually said.
Have someone else run some interviews. Founders unconsciously steer toward their own hypothesis. A co-founder or colleague following the same guide will surface different things, and the differences are informative on their own.
Actively hunt for disconfirming cases. Deliberately seek out people who solved this problem another way and are happy, or who considered a tool like yours and chose not to buy. They are more useful than another enthusiast.
Count instead of remembering. Tally how many people described a specific past attempt, spent money, or agreed to a commitment. A count is harder to argue with than a recollection.
Validation is not a phase you finish before building. The teams that keep learning are the ones that run this continuously, feeding what they hear back into the roadmap through a standing customer feedback loop. Once you have a product in the market, the question shifts from whether people want it to whether they keep using it, which is the subject of how to find product market fit without fooling yourself. If you plan to run the conversation side with AI moderation, qualitative research with AI covers where that works and where it does not.
Frequently Asked Questions
How many customers should I talk to before building?
Between 15 and 20 within a single clearly defined segment is a reasonable plan, and the real stopping rule is saturation: when two consecutive interviews produce nothing you have not already heard. If you are still learning something new at conversation 25, your segment is too broad and you are effectively studying several markets at once.
What is the Mom Test?
It is a rule from Rob Fitzpatrick's book of the same name: ask questions your mother could not lie to you about. That means asking about facts in the person's life rather than opinions about your idea. "How did you handle this last month" passes the test. "Do you think this is a good idea" fails it, because the answer is shaped by politeness rather than by the truth.
Should I show people my product during a discovery interview?
Not until the end, and only if you need to. Once you describe the product, the person switches from reporting their experience to reacting to yours, and the reactions are much less reliable than the reports. Get the history of the problem first, then demo if you want feedback on the solution, and treat the two parts of the call as separate studies.
Is it a problem that I am talking to people I know?
It skews everything toward encouragement, because people who like you want you to succeed. Use your network for introductions rather than for interviews, so that everyone you actually talk to is one step removed from you. If you must interview a contact, be explicit that unhelpful truth is what you need, and weight their enthusiasm at close to zero.
How do I validate an idea in a market I do not know?
Recruit externally rather than through your network, screen on recent behaviour rather than job title, and expect to spend more time on recruitment than on the interviews themselves. Consider running a small number of depth conversations to learn the vocabulary, then a larger structured study to check whether what you heard generalises, since your intuition about which anecdotes are typical is unreliable in an unfamiliar market.
What counts as real validation?
A commitment with a cost attached: money, a signed pilot, a scheduled meeting with the budget holder, an introduction to a colleague, time spent configuring something. Everything else, including strong verbal enthusiasm and email signups, is interest. Interest is worth collecting and it is not evidence.
