Retour
Causal Research: Definition, Designs and Examples

Matthieu Saussaye

Causal research tests whether a change in one variable produces a change in another, and by how much. It isolates a cause, such as a price, a message or a feature, by changing it deliberately for some people and not others, usually through a randomized experiment or A/B test, then comparing outcomes. It answers: if we do X, will Y change?
The short version:
Three conditions must hold. The cause comes before the effect, the two move together, and no third factor explains the link.
Randomization is the strongest tool. Randomly assigning people to versions makes the groups alike on everything except the change you made.
Correlation is not enough. Customers who use a feature may renew more because they were already your most engaged customers, not because of the feature.
Business forms are everyday: A/B tests on websites, price tests, ad holdouts, split-sample survey experiments and monadic concept tests.
When you cannot randomize, quasi-experiments such as matched control regions or before-and-after with a comparison group get you part of the way.
It needs a hypothesis. You test a specific claim, which usually comes from exploratory or descriptive work done earlier.
Causal research is the most demanding of the purpose-based types of research. This guide shows how to design one, which design to choose when a pure experiment is impossible, and how surveys can run experiments too.
Causal research designs compared
Designs differ in how confidently they rule out other explanations. Researchers call that internal validity.
Design | How it works | Confidence in cause | Business example | Main weakness |
|---|---|---|---|---|
Randomized controlled experiment | People randomly assigned to treatment or control | High | Two onboarding flows, random 50/50 split | Can be slow or impossible for big decisions |
A/B test (online) | Randomized experiment run on live traffic | High for the measured metric | Two checkout page versions | Short-term metrics may not reflect long-term effects |
Split-sample survey experiment | Each respondent sees one randomly assigned version of a question, price or concept | High for stated reactions | Monadic price test: each group sees one price | Measures stated intent, not real purchase |
Quasi-experiment | Treatment and comparison groups not randomly assigned, such as test and control regions | Medium | New price in two regions, compare with similar regions | Groups may differ in ways that also affect the outcome |
Before-and-after without control | Measure, change something, measure again | Low | Sales before and after a campaign | Seasonality and other events are mixed into the effect |
Observational analysis | Statistical models on existing data, controlling for known factors | Low to medium | Regression of churn on feature usage | Unmeasured confounders can still drive the result |
Move down the table only when the row above is impossible, and say in your report which design you used.
The three conditions for causality
Every causal claim rests on the same three tests. Fail any one and you have an association, not a cause.
Temporal precedence. The cause happens before the effect. If satisfaction scores rose before the new support team started, the team cannot be the cause.
Covariation. When the cause changes, the effect changes too. Customers exposed to the new message must behave differently from those who were not.
No plausible alternative explanation. Nothing else changed at the same time that could explain the difference. This is the hard one, and the reason randomization matters.
In epidemiology, where experiments are often unethical, Sir Austin Bradford Hill set out nine viewpoints in 1965 for judging whether an observed association is likely to be causal, including strength, consistency, temporality, a dose-response gradient and experimental evidence (Bradford Hill criteria). They are a useful checklist for business analysts working with observational data too.
The confounder problem, with an example
Suppose customers who attended an onboarding webinar renew more often. Three stories fit that pattern:
The webinar teaches people to get value, so they renew. (Causal.)
Customers who were already committed are the ones who sign up for webinars. (Self-selection.)
Larger accounts both attend webinars and renew more, because they have dedicated admins. (Confounding by account size.)
Analytics cannot separate these. Randomly inviting half of new accounts to the webinar can: if the invited half renews more, the webinar is doing the work.
How to conduct causal research in 7 steps
State one hypothesis with a direction and a size. 'Showing annual pricing first increases annual-plan uptake by at least 3 points.'
Define the treatment exactly. What changes, for whom, from when. Everything else stays identical.
Pick one primary outcome. Decide in advance which metric decides the test. Picking the best-looking metric afterwards inflates false positives.
Randomize the assignment. Use a random process, not 'odd customer IDs' or 'the Monday cohort', which can hide patterns.
Size the test. Small effects need large groups. Calculate the sample needed to detect the smallest effect worth acting on before you start.
Run it long enough. Cover full weekly cycles and give novelty effects time to fade.
Analyze, then explain. Report the effect with a confidence interval. If the effect is real, use explanatory research to find out why it worked, which tells you what to try next.
Causal research in surveys
Surveys alone do not prove causes when you simply ask people why they did something. People rationalize. But a survey becomes a true experiment once you randomize what each respondent sees.
Monadic concept tests. Each respondent sees one concept only, randomly assigned. Differences in appeal between groups reflect the concepts, not the order they were shown in. See concept testing for the full method.
Price experiments. Randomly assign different price points to comparable groups and compare purchase intent.
Message tests. Show one of several headlines or claims, then ask the same outcome questions to all groups.
Question wording experiments. Randomly vary wording to measure how framing changes answers.
Conjoint analysis. Respondents choose between product profiles with randomly varied attributes, letting you estimate the effect of each attribute on choice.
The limit: survey experiments measure stated reactions. A price that wins in a survey still needs a market test before you change your price list.
Outcome questions for survey experiments
How likely would you be to buy this product at the price shown? (5-point scale)
How appealing is this offer to you? (very unappealing to very appealing)
How believable is this claim? (not at all to completely)
Which of these would you choose if you were buying today? (profiles A, B, C, none)
How much would you expect to pay for this? (open numeric)
Ask the same outcome questions, in the same order, to every group. Only the stimulus should differ.
A real example: the Bing headline test
Online experiments are the purest business form of causal research. In a 2017 Harvard Business Review article, Ron Kohavi and Stefan Thomke describe a 2012 idea at Microsoft's Bing to change how ad headlines were displayed. Judged low priority, it sat in the backlog for more than six months until an engineer ran a simple A/B test. Within hours the new version triggered a “too good to be true” alert. It was not a bug: the change had raised revenue by 12%, which they estimate at more than $100 million a year in the United States alone, without hurting user-experience metrics (HBR, 2017).
The lesson is not that every test wins. It is that the team would never have known the size of the effect, or trusted it, from opinions or correlations. A randomized comparison turned a hunch into a measured cause.
Causal vs explanatory vs descriptive research
The terms overlap in textbooks, and some use causal and explanatory interchangeably. In practice they do different jobs.
Descriptive | Explanatory | Causal | |
|---|---|---|---|
Question | What is happening? | Why and how does it happen? | Does X cause Y, and by how much? |
Main evidence | Measurements | Patterns, drivers, mechanisms | Controlled comparison |
Typical method | Survey, analytics | Driver analysis, follow-up interviews | Randomized experiment |
Output | Size and profile | A plausible mechanism | An effect size with a confidence interval |
Common mistakes in causal research
Peeking and stopping early. Checking results daily and stopping the moment a difference looks significant inflates false positives. Fix the duration or sample size in advance.
Testing too many things at once. If the new page changes the headline, the image and the price, a win tells you nothing about which change mattered.
Too small a sample. An underpowered test will often show 'no difference' even when a worthwhile effect exists.
Contaminated groups. If control customers hear about the treatment offer from colleagues, the comparison is no longer clean.
Measuring only the short term. A discount can lift this week's conversion and lower next quarter's margins. Track a downstream metric too.
Generalizing beyond the test. An effect measured on new website visitors in one country may not hold for existing customers elsewhere.
Advantages and limitations of causal research
Advantages | Limitations |
|---|---|
The only design that shows what an action will do | Requires a clear hypothesis up front |
Gives an effect size you can put in a business case | Can be slow, costly or politically hard to run |
Removes opinion from contested decisions | Lab and survey settings may not match real life |
Repeatable: the same test can be rerun | Shows that something works, not why it works |
Online tests can be cheap at scale | Ethical limits on what you can randomly withhold |
Find out why your winning version won
A causal test tells you which version performed better. It rarely tells you what respondents noticed, believed or disliked, which is what you need to design the next version.
SmartInterview can run split-sample survey experiments, with each respondent randomly shown one concept, price or message, and then ask open follow-up questions by voice or text with AI probing on vague answers, in the respondent's language. Open answers are coded into themes automatically, so you can compare the reasons behind each version's score.
Start free and set up your first survey experiment.
Frequently Asked Questions
What is causal research in simple terms?
Causal research tests whether one thing causes another. You change one factor for some people and not for others, ideally assigned at random, and measure whether the outcome differs. If it does, and nothing else differs between the groups, the change caused the difference.
What is an example of causal research?
An A/B test that randomly shows half of website visitors a new checkout page and compares completed purchases is causal research. So is a survey where each respondent is randomly shown one of three prices and purchase intent is compared across the groups.
What are the three conditions for causality?
The cause must come before the effect, the cause and effect must vary together, and there must be no plausible alternative explanation such as a confounding variable. Randomized experiments are designed to satisfy the third condition.
Is causal research qualitative or quantitative?
Causal research is quantitative. It relies on controlled comparisons between groups and statistical tests of the difference. Qualitative research can suggest likely causes and explain mechanisms, but it cannot measure the size of an effect.
What is the difference between causal and correlational research?
Correlational research measures whether two variables move together without changing either one. Causal research deliberately changes one variable and controls other factors, so it can show that one variable produces the change in the other.
Can a survey establish causation?
A survey can establish causation when it includes an experiment, meaning respondents are randomly assigned to different versions of a stimulus and their answers are compared. Asking respondents why they did something does not establish causation, because people often misreport their reasons.


