Back
Qualitative Data: How to Collect, Code and Report It

Matthieu Saussaye

The Short Answer
Qualitative data is non-numeric evidence: what people said, wrote or did, in their own words. It explains why something happens. Quantitative data measures how much.
Sources: interviews, open survey responses, focus groups, observation, support tickets, reviews.
Method: read, code into themes, build a codebook, check agreement between coders.
Output: named themes, their relative weight, and quotes that trace back to the raw text.
You can count themes. You cannot project them onto a population like a survey percentage.
Qualitative vs Quantitative
The two are not rivals. They answer different questions, and a study that uses only one of them usually has a hole in it.
Dimension | Qualitative | Quantitative |
|---|---|---|
Question it answers | Why, how, in what context | How many, how often, how much |
Data | Words, transcripts, images, observed behavior | Numbers, scale points, counts |
Typical sample | Small and purposive, often 10 to 40 | Large and representative, often hundreds |
Stopping rule | Saturation: new interviews stop producing new themes | A target margin of error |
Analysis | Coding into themes, interpretation | Statistical summary and significance testing |
Main risk | Reading your own assumptions into the text | Measuring precisely the wrong thing |
Best used to | Generate hypotheses, explain a number that moved | Test a hypothesis, size a market, track over time |
The usual sequence is qualitative first to work out what the questions should be, quantitative to size the answers, then qualitative again to explain whatever the numbers refused to explain. A fuller treatment of the trade-offs is in qualitative vs quantitative research.
Where Qualitative Data Comes From
Depth interviews
One person, one conversation, usually 30 to 60 minutes with a loose guide rather than a script. This is the richest source because you can follow the thread. The cost is time: recruiting, scheduling, running, transcribing and analyzing 20 interviews is weeks of work in a traditional setup.
Open survey responses
The most abundant and most wasted source in most companies. A single open question attached to a rating scale produces thousands of comments a year, and in most organizations they are read once, cherry-picked for a slide, and never coded. Open responses are shallower than interviews because nobody asks a follow-up, but they arrive at survey scale and cover the same population as your numbers, which makes them the natural place to look when a metric moves.
Focus groups
Six to ten people discussing a topic together. Useful for reactions to stimulus, language and terminology, and watching a group build a shared opinion. Weak for anything sensitive or individual, because the loudest participant sets the frame and the group converges. Never treat a focus group as a small survey.
Observation
Watching what people actually do, in a usability session or in their own environment. This catches the gap between claimed and real behavior that no self-report can reach. People are unreliable narrators of their own habits, not because they lie but because routine is invisible from the inside.
Support tickets, reviews and sales notes
Free, continuous and already in your systems. The catch is severe selection bias: tickets come from people with problems, reviews from the delighted and the furious. Excellent for finding out what can go wrong, useless for estimating how common anything is. Treat this data as a source of hypotheses, never as a measure of prevalence.
The Coding Process
Coding is the step that turns a pile of text into something you can analyze. Four stages.
1. Open coding
Read a subset of the data, typically 10 to 20 percent, and label each meaningful passage with a short descriptive tag. Stay close to the words on the page. If a respondent says the export "took three tries", code it as something like "export fails repeatedly", not as "poor UX", which is your interpretation rather than their statement. Expect 40 to 80 raw codes at this stage and expect them to be messy.
2. Build the codebook
Group the raw codes into themes and write the codebook: for each theme, a name, a one-sentence definition, an explicit inclusion rule, an exclusion rule, and two example quotes. The exclusion rule is the part people skip and the part that does all the work, because it is what stops a theme like "pricing" from quietly swallowing every comment that mentions money.
Aim for 8 to 15 top-level themes. Fewer and everything ends up in a bucket called "other". More and no one can hold the structure in their head. Add a second level underneath where a theme is doing too much: "pricing" splits usefully into "price too high", "pricing model confusing" and "unexpected charges", which are three different problems for three different teams.
3. Apply the codebook to everything
Code the full dataset against the codebook. Allow a passage to carry more than one code, because comments genuinely contain more than one idea. When something does not fit, do not force it: log it, and if the same misfit appears several times, that is a new theme and you re-code the earlier material to match. A codebook that never changes during coding is usually a codebook nobody is really applying.
4. Check inter-coder agreement
Have a second person independently code a sample of the material, commonly 10 to 20 percent, then compare. Where you disagree, the fault is almost always an ambiguous definition rather than a careless coder, so the output of the exercise is a sharper codebook, not a scolding. Re-code and repeat until two people reading the same comment reach the same theme most of the time.
If you are the only analyst, code a sample, wait a day, and code it again blind. Disagreeing with yourself is uncomfortable and highly informative.
Counting Themes Without Faking Statistics
Once coded, you can count. The counting is legitimate. The projection is where people get into trouble.
What you can honestly say:
"Of the 412 open comments left by detractors, 96 mention onboarding, making it the largest single theme."
"Onboarding was the most frequent theme this quarter, up from third last quarter."
"Every one of the 14 enterprise customers we interviewed raised the approval workflow unprompted."
What you cannot say:
"23% of customers have an onboarding problem." Only the ones who wrote a comment are in the base, and they are not a random sample of customers.
"Onboarding complaints are up 40%." Not unless comment volume was stable, and it never is.
"Only 3% mentioned security, so it does not matter." Silence in an open question means nobody thought to raise it, not that nobody cares. Absence of mention is not evidence of absence.
Three rules keep the counting honest. Always state the base, and make clear it is the number of comments or interviews, not the number of customers. Report shares of coded material, not shares of the population. Compare like with like, meaning the same question, same channel and same coding scheme across periods, otherwise a trend line is measuring your own process changes.
Frequency is also not the same as importance. A theme mentioned by 9 people who are collectively half your revenue outranks one mentioned by 90 who churned last year anyway. Report frequency and weight, and keep them in separate columns so nobody quietly merges them.
The Traceability Rule
Every theme must link back to the raw words that produced it. This is the single discipline that separates qualitative analysis from opinion with citations.
In practice it means that when your report says "onboarding friction, 96 mentions", someone should be able to click that theme and read all 96 comments, in full, with enough context to see who said them. Not three quotes chosen by the analyst. All of them.
Traceability does three things. It makes the analysis auditable, so a challenge in a meeting is settled by reading rather than by seniority. It exposes coding drift, because a theme whose contents no longer match its label becomes obvious the moment you read the list. And it changes the politics of the room: a stakeholder who disagrees with a theme can go and read it, which converts an argument about credibility into a discussion about interpretation.
Whatever your tooling, keep the identifier of each source passage attached to its code from the first day. Retrofitting traceability onto a spreadsheet of counts is not possible, and this is why so many qualitative decks cannot survive a direct question.
Reporting So It Survives the Meeting
Qualitative findings lose arguments to charts, not because they are weaker evidence but because they are usually presented worse. Fix the presentation.
Lead with the finding, not the method. "Customers abandon setup at the data import step" is a finding. "We conducted 22 interviews" is a preamble. Method belongs in an appendix that you are glad to have when someone asks.
Name themes in the respondents' language. "I can't tell if it saved" beats "perceived system feedback deficiency". The first one gets quoted back to you by an engineer a week later.
Show the size of each theme, with its base. A simple ranked bar of themes with counts and a visible denominator answers the "is this just one loud customer?" question before it is asked.
Give two quotes per theme, not eight. One typical, one sharp. Link to the rest rather than pasting them.
Pair each theme with the number it explains. Qualitative evidence is at its most persuasive when it is answering a quantitative question someone already cares about, such as why a metric dropped in one market.
Say what is not in the data. Who you did not speak to, and what that means. Stating limits yourself is far stronger than having them found.
End with decisions, not observations. Each theme should carry a recommended action and an owner, or it becomes a slide that gets admired and forgotten.
Where AI Fits, and Where It Does Not
Automated coding has changed the economics of this work. A model can apply a codebook to thousands of open responses in minutes, a job that used to consume analyst weeks and was therefore usually skipped entirely. It can also propose a first-pass code frame from the text, which is a genuinely useful starting point.
Two things do not change. A human still has to own the codebook, because the definitions encode what your organization means by a theme and that is a judgment call. And traceability matters more, not less: if a model assigns a theme, you need to be able to read every passage it assigned and confirm they belong together. Automated coding without an audit trail is a confident summary you cannot check, which is worse than no coding at all. Qualitative research with AI goes through the workflow in detail.
Related Reading
Turn open responses into evidence you can count
Most teams already own more qualitative data than they can process. The comments arrive, they are skimmed, two of them end up in a slide, and the rest sit in a spreadsheet nobody opens again.
SmartInterview closes that gap at both ends. Surveys run by voice or text with an AI that asks the follow-up a good interviewer would have asked, so you get interview-quality material at survey scale, in the respondent's own language. Those responses are then coded automatically into themes you can count, and every theme stays linked to the exact words behind it, so any finding can be opened and read rather than taken on trust.
Point it at one question you already ask and see what comes back.
Frequently Asked Questions
What is qualitative data?
Qualitative data is non-numeric evidence about what people think, say and do, captured in their own words or through observation. Interview transcripts, open survey responses, support tickets and field notes are all qualitative data. It describes reasons, context and meaning rather than quantity.
What is the difference between qualitative and quantitative data?
Quantitative data is counted or measured and answers how many, how often and how much. Qualitative data is described and interpreted, and answers why and how. Quantitative work needs a representative sample to be projectable; qualitative work needs a purposive sample and stops when new interviews stop producing new themes.
What are examples of qualitative data?
Interview and focus group transcripts, open-ended survey answers, customer support conversations, product reviews, sales call notes, usability session recordings, diary entries and observational field notes.
How do you analyze qualitative data?
Read a subset and label passages with descriptive codes, group those codes into 8 to 15 themes and write a codebook with inclusion and exclusion rules, apply the codebook to the full dataset, and have a second person independently code a sample to check agreement. Then count theme frequency against a clearly stated base and keep every theme linked to its source passages.
Can qualitative data be quantified?
You can count how often a theme appears and compare that across periods or segments, as long as the base is the number of comments or interviews rather than the population. What you cannot do is turn theme frequency into a population percentage, because the people who wrote a comment are not a random sample of your customers.
How many interviews are enough?
Until saturation: the point at which additional interviews stop producing themes you have not already recorded. For a narrow question with a homogeneous audience that often arrives early; for a broad question across several distinct segments you need enough interviews within each segment, so the total climbs quickly.
What is inter-coder agreement and why does it matter?
It is the extent to which two people independently assign the same code to the same passage. It matters because it is the only real evidence that your themes exist in the data rather than in the analyst's head. Low agreement is usually a symptom of vague theme definitions, which you fix by sharpening the codebook and re-coding.


