"We ran the survey, but we don't know how to analyze the results." A common pain. As response volume grows, staring at Excel reveals nothing.
This article covers the fundamentals of survey analysis and tool selection by situation.
Three analysis levels
| Level | What you do | Example tools |
|---|---|---|
| Lv1: Aggregation | Simple counts (per-question response ratios) | Excel |
| Lv2: Cross-tab | Compare by attribute / segment | Excel, BI tools |
| Lv3: Open-text + inference | Text classification, statistical tests, prediction | AI, stats software |
Different tools shine at different levels.
Lv1: Simple aggregation
What to look at
- Response ratios per question (pie / bar chart)
- Mean, median (for 5-point scales)
- Standard deviation (variance)
Excel example
Q-A (5-point satisfaction):
Value | Count | %
1 (dissatisfied) | 5 | 5.0%
2 | 12 | 12.0%
3 (neutral) | 28 | 28.0%
4 | 35 | 35.0%
5 (satisfied) | 20 | 20.0%
Total |100 |100.0%
Mean: 3.53
Median: 4
SD: 1.10
COUNTIF() alone handles this.
Watch out for
- Don't look only at the mean — check distribution (centered vs. polarized?)
- Decide whether to exclude "no response"
Lv2: Cross-tabulation
What to look at
- By attribute (industry, role, etc.)
- Time-series change
- Inter-question relationships
Excel pivot tables
The basic cross-tab tool:
| Satisfied | Neutral | Dissatisfied
Industry A | 60% | 30% | 10%
Industry B | 40% | 35% | 25%
Industry C | 50% | 40% | 10%
Surfaces "Industry B alone is low" in one view.
BI tools (Tableau, Looker, Power BI)
Past a few thousand responses, Excel gets sluggish. BI tools are dramatically faster with real-time filters.
Choosing cross-tab axes
Don't cross blindly — pick axes from hypotheses.
Good axes:
- Customer plan × satisfaction
- Tenure × NPS
- Industry × usage frequency
- Hire cohort × engagement
Too many combinations turn into "everything goes" analysis — meaningless.
Lv3: Open-text analysis
Quantitative approach: text classification
Classify open text into themes and count:
100 open-text responses from a CSAT survey:
Theme | Count | %
Pricing / cost | 28 | 28%
Support quality | 22 | 22%
Features / usability | 18 | 18%
Performance | 12 | 12%
Other | 20 | 20%
Improvement priorities become quantified.
Manual vs. AI
Up to ~10 responses, manual is fine. Past 50, AI auto-classification is far more efficient.
Repoan's report feature auto-classifies open text via AI and summarizes per-theme trends.
Qualitative approach: narrative analysis
Read responses as stories, not numbers:
- The phrases the customer used
- Emotional arcs (joy / anger / resignation)
- Context shared across multiple responses
Useful for understanding "why they feel that way" — invisible in numbers.
Confirming statistical significance
Before claiming "Industry A and B differ on satisfaction," confirm the difference isn't random noise.
Common tests
- Chi-squared — significance of proportional differences (satisfied / dissatisfied)
- t-test — significance of mean differences (5-point average)
- ANOVA — 3+ group mean comparisons
All available in Excel (CHISQ.TEST, T.TEST).
Sample size sufficiency
- Under 30: tests are unreliable
- ~100: large effects detectable
- 500+: small effects detectable
At low sample sizes, frame as "looks like a trend" — don't make definitive claims.
Report-writing tips
Top-down: "conclusion → evidence → detail"
Lead with conclusion in the exec summary; evidence underneath; full data in appendix.
One slide = one message
"X drives Y to be low" — one message per slide. Information overload doesn't communicate.
Quote open text directly
Beyond aggregates, quote specific customer voices. Numbers + stories move decision-makers.
NPS: +12 (-8 vs. last period)
Promoter voice:
> "Support is fast, follow-through is consistent"
Detractor voice:
> "Higher pricing than industry, slow feature pace"
What not to do
❌ Open text pasted in full, unanalyzed
Decision-makers won't read it. Classify and present with counts.
❌ Definitive claims on small-segment samples
"All 3 respondents from Industry X were dissatisfied" — likely random.
❌ Mean only
Mean 3.5 on 5-point can be "centered" or "polarized" — very different stories. Always show distribution alongside.
❌ Cherry-picking
Only positive data → internal trust collapses. Always include negative findings.
Summary
Analysis order:
- Simple aggregation for overall picture
- Cross-tabs for hypothesis-driven differences
- Open-text analysis for "why"
- Statistical tests for significance
- Conclusion-first in the report
Repoan's AI report feature handles Lv1–Lv3 in one click:
- Simple aggregation: auto-generated ratios, means, distributions with charts
- Cross-tabs: per-segment (industry, plan, tenure) comparison tables
- Open-text classification: theme aggregation + representative-voice extraction
- Improvement proposals: prioritized actions from the analysis
- Exec summary: 1-page summary auto-generated for leadership
- PDF export: editable report deliverable as URL or PDF
100 open texts by hand: 4–6 hours. AI report: 30 seconds (details). After generation, edit in the markdown editor — "AI draft, human polish" is the most efficient workflow.
Related reading
- Numbers don't tell you customers: data-driven customer understanding
- Combining quant and qual: the quant×qual workflow
- Operationalizing results: putting survey results to work