The short answer
Start Product Discovery with the evidence your decision is missing. Analytics helps establish where behavior changed and for whom. Interviews investigate the circumstances of a choice. An experiment tests whether a proposed change affects an outcome. The right sequence depends on the question; having a dashboard does not make quantitative analysis a mandatory first step.
Choose your first step
| Situation | First step | What it cannot establish |
|---|---|---|
| Activation falls in an existing product | Check events, denominators, windows and traffic composition | Why an individual declined to proceed |
| A new market with no users yet | Research Jobs, existing alternatives and decision context | Paying demand for your particular solution |
| The problem is understood; solutions are disputed | Identify the riskiest assumption and choose a test | Long-term impact from a successful prototype alone |
Worked example: conversion rises from 32% to 52%
Illustrative example: these invented numbers demonstrate a calculation and do not describe my project. Activation means a new account completing a target action within seven days. We compare two fully observed weekly cohorts with unchanged event definitions.
| Source | Before: accounts / activated | After: accounts / activated |
|---|---|---|
| A | 200 / 160 = 80% | 500 / 410 = 82% |
| B | 800 / 160 = 20% | 500 / 110 = 22% |
| Total | 1,000 / 320 = 32% | 1,000 / 520 = 52% |
The overall increase is 20 percentage points, but conversion within each source rises by only two points. Applying the new rates to the original traffic mix gives 0.2 × 82% + 0.8 × 22% = 34%.
Using the original period’s weights, 34% − 32% = 2 pp comes from within-source rate changes; 52% − 34% = 18 pp comes from changing source shares at the new rates. This is an exact arithmetic decomposition for this chosen order, not a causal attribution between product and marketing. A different decomposition order may allocate the interaction differently.
The table does not show that a new feature caused a 20-point improvement. Even the two-point increase within groups could reflect seasonality, audience changes or chance. A causal conclusion requires an appropriate research design.
Check the data before interpreting it
- Define the analysis unit: person, account or organization. Do not silently substitute sessions.
- Use comparable observation windows and exclude immature cohorts.
- Check event loss, duplication, bot filtering and changes to analytics consent.
- Separate changes in audience composition from changes within comparable groups.
- Record sample sizes and uncertainty. An inconclusive difference does not establish equivalence.
Turn the signal into an interview question
In this example, investigate both activated and nonactivated accounts from each source. This sampling supports comparison, but it does not turn four behavioral groups into Job-based segments. A segment needs evidence of shared Jobs and relevant circumstances, not just an acquisition label.
Ask participants to reconstruct a recent episode: what happened before they looked for a solution, what outcome they wanted, what they had already tried, how they chose and where they stopped. Avoid embedding your preferred explanation in the question. Record contradictory cases and recruitment nonresponse; people who agree to talk may differ from those who do not.
Ivan Zamesin’s AJTBD canon examines desired transitions and connected Jobs. A past-experience interview helps investigate that structure. It does not establish how prevalent the discovered problem is.
Choose a test for the proposed solution
Identify the risk first. Do people see value, understand the interface, and can the team deliver a viable business solution? SVPG’s four risks help separate these questions. A prototype can test comprehension; estimating a causal activation effect generally calls for an experiment with appropriate assignment.
Before launch, specify the randomization unit, primary metric, smallest useful effect, duration and decision rule. Guardrails could include refunds, support contacts and contribution margin. When randomization is unavailable, state the limitations of the observational comparison.
Teresa Torres’s Continuous Discovery helps teams maintain customer contact and test assumptions. A regular habit is useful; interview count alone does not measure decision quality.
A decision record you can reuse
- Decision: what will we change, defer or stop?
- Observation: metric, denominator, period, source and sample size.
- Explanations: working hypothesis and alternatives, including data errors.
- Missing evidence: customer context, prevalence, usability, impact or economics.
- Test: method, sample, timeframe, criterion and owner.
- Result: what we learned, what remains uncertain and what happens next.
This is my working template, not a promise of growth. For further approaches, see 15 authors on product management and growth.
Discuss your next step
Looking for a perspective on a growth challenge? Let's talk.