Theme Extractor
Text Analysis
NLP
Automatically discover hidden themes and topics in customer reviews, survey responses, and open-ended feedback using machine learning.
OVERVIEW & WHAT THIS TOOL SOLVES
Reading 500 open-ended survey responses or product reviews by hand takes days, while simple word clouds just highlight repetitive filler words ("coffee", "product", "good") that appear in every review. The Theme Extractor converts raw customer text into a TF-IDF document-term matrix and applies Non-Negative Matrix Factorization (NMF) to automatically cluster distinctive vocabulary into coherent latent topics and assign every document to its primary theme.
🎯 Questions It Answers
- What are the 2 to 8 recurring latent themes hidden across hundreds of customer reviews, social comments, or support tickets?
- What percentage of documents belong primarily to each discovered theme, and which top keywords define each topic?
- How does theme prevalence shift across customer segments, product tiers, or competitors?
📂 What Data You Need
- Text Corpus: Preset marketing text datasets (influencer comments, streaming feedback, product reviews) or uploaded CSV with a text column
- Topic Count & Stopwords: Number of themes
K(or automatic selection) and custom domain stopwords to filter out - Optional Grouping Column: Segment, brand, or rating column to compare theme shares across groups
💡 What You'll Discover
- Theme Cards & Anchor Vocabulary: Top weighted terms, primary document counts/shares (summing to 100%), and representative quotes
- Document-Level Topic Loadings: Continuous theme weights for every row plus primary-theme classification
- Cross-Group Theme Breakdown: Comparative bar charts showing which segments over-index on specific complaints or praise
🔍 Key Nuances & Common Traps to Watch For
- Why TF-IDF Beats Raw Word Counts: If you analyze coffee subscription reviews, the word "coffee" appears in 95% of rows. Inverse Document Frequency (IDF) down-weights ubiquitous category words so distinctive terms ("grind", "billing", "roast", "late") drive the themes.
- Bag-of-Words Ignores Word Order & Negation: NMF treats documents as unordered term vectors ("not fast" and "very fast" share "fast"). Always read the top representative verbatim quotes on each theme card before naming a topic.
- Choosing the Number of Themes (K): Too few themes merge distinct issues into broad buckets; too many themes split a single topic into redundant synonyms. Test
K = 3to6and inspect vocabulary overlap.
📐 Under the Hood: Model Equations & Variable Definitions
1. Term Frequency–Inverse Document Frequency (TF-IDF) Weighting
\[ \text{tfidf}_{d,t} = \text{tf}_{d,t} \times \left(\ln\!\left(\frac{1 + D}{1 + \text{df}_t}\right) + 1\right), \qquad V_{d,t} = \frac{\text{tfidf}_{d,t}}{\sqrt{\sum_{m} \text{tfidf}_{d,m}^2}} \]
💬 Read it in plain English: A word t gets a high weight V_{d,t} in document d when it appears frequently inside that specific review (high tf_{d,t}) but rarely across the rest of the D reviews (low document frequency df_t), normalized so long reviews don't dominate short ones.
D, d, t
Corpus Size, Document & TermYOUR DATA ⓘ
Total documents D, individual document index d, and vocabulary term index t.
tfd,t, dft
Term & Document FrequenciesYOUR DATA ⓘ
Count of term t in document d and number of documents containing term t at least once.
Vd,t
Normalized TF-IDF Matrix EntryOUTPUT ⓘ
L2-normalized importance weight of term t in document d (forming the D × M matrix V).
2. Non-Negative Matrix Factorization (NMF) & Primary Theme Assignment
\[ \min_{\mathbf{W} \ge 0,\; \mathbf{H} \ge 0} \|\mathbf{V} - \mathbf{W}\mathbf{H}\|_F^2 = \sum_{d=1}^{D}\sum_{t=1}^{M} \left(V_{d,t} - \sum_{k=1}^{K} W_{d,k} H_{k,t}\right)^2, \qquad k^*(d) = \arg\max_{k} W_{d,k} \]
💬 Read it in plain English: NMF factors the D×M TF-IDF matrix V into two non-negative matrices—W (how strongly each document loads on each of the K themes) and H (how strongly each word defines each theme)—and assigns each document to its highest-loading theme k*(d).
K
Number of Latent ThemesYOUR DATA ⓘ
Target number of topics extracted by the factorization.
W (Wd,k)
Document-to-Theme Loading MatrixESTIMATED ⓘ
Non-negative weight (D × K) measuring how much document d expresses theme k.
H (Hk,t)
Theme-to-Word Basis MatrixESTIMATED ⓘ
Non-negative weight (K × M) ranking the top anchor words that define theme k.
k*(d)
Primary Document ThemeOUTPUT ⓘ
Dominant theme assigned to document d, used to compute each theme’s % share of documents.
DATA SOURCE
📚
Use a Case Study
📊 Pre-loaded Text Datasets
Select a preset scenario to explore real-world text data with theme extraction. Each scenario includes sample reviews, comments, or survey responses.
📤
Enter Your Data
Max 500 documents Max 5,000 chars each
Paste your text data below. Each line will be treated as a separate document (review, comment, etc.).
0 documents
0 characters
Upload a CSV file with a text column. The tool will auto-detect columns containing text.
Drag & Drop CSV file (.csv, .tsv, .txt, .xls, .xlsx)
Upload a CSV with a text column. The tool will auto-detect text columns.
No file uploaded.
ANALYSIS SETTINGS
Analysis Settings
Auto-detect finds the optimal number based on your data.
More words give richer theme descriptions.
Choosing the Right Number of Themes
📊 Auto-Detection Method
When set to "Auto-detect," the algorithm tests different theme counts (2-8) and selects the one with the best coherence score. Coherence measures how semantically similar the top words in each theme are to each other.
- High coherence: Theme words relate to a single concept (e.g., "battery, charge, power, life")
- Low coherence: Theme words are scattered (e.g., "battery, shipping, color, return")
📈 When to Override Auto-Detection
| Situation | Recommended Action |
|---|---|
| Themes seem too broad/general | Increase theme count (more specific topics) |
| Similar themes detected | Decrease theme count (consolidate) |
| You know the domain well | Set count based on expected categories |
| Very small dataset (<100 docs) | Use 2-4 themes to avoid overfitting |
| Large, diverse dataset (>500 docs) | Try 5-8 themes for granularity |
💡 Iterative Approach
Topic modeling often benefits from experimentation:
- Start with auto-detect to get a baseline
- Review the themes—do they make business sense?
- If themes overlap, reduce count; if too broad, increase count
- Compare sample comments across runs to validate
Words per Theme: Quality vs. Specificity
📊 What This Controls
This setting determines how many top-weighted words are shown for each discovered theme. The words are ranked by their importance to that theme (NMF weight).
📈 Trade-offs
| Words | Pros | Cons |
|---|---|---|
| 5 words | Clean, focused, easy to label | May miss nuances |
| 8 words (default) | Good balance of clarity and detail | — |
| 10-12 words | Richer context, better for unfamiliar domains | Can include lower-relevance terms |
💡 Recommendation
Start with 8 words (default). Increase to 10-12 if you're exploring unfamiliar data and need more context to understand themes. Decrease to 5 for clean presentation in reports.
Sample Comments by Theme
Most representative documents for each theme (highest confidence scores).
Understanding Sample Comments & Confidence Scores
📊 What Are Sample Comments?
These are the most representative documents for each theme—the ones the algorithm is most confident about assigning to that theme. They serve as concrete examples of what each theme "looks like" in your data.
📈 Understanding Confidence Scores
The confidence percentage reflects how strongly a document matches its assigned theme:
💡 How to Use Samples
⚠️ When Samples Don't Match Expectations
If sample comments seem wrong for a theme: