🔬 Zero-Shot BART & Lexical Quant Lab
Transform unstructured qualitative reviews (Yelp, Amazon, DTC surveys) into rigorous quantitative variables. Score every document against custom business constructs via Zero-Shot Natural Language Inference (BART-MNLI), identify statistically distinctive group vocabulary with Monroe–Colaresi–Quinn Fightin' Words Dirichlet log-odds, and regress star ratings or retention on your construct scores.
OVERVIEW & WHAT THIS TOOL SOLVES
Traditional sentiment dictionaries only tell you whether a customer review uses positive or negative words—they cannot tell you which operational failure caused a 1-star rating when a diner praises the food yet complains about a 45-minute seating wait. The Zero-Shot BART & Lexical Quant Lab transforms unstructured customer reviews into quantitative variables by scoring each document against your custom business constructs via Transformer Natural Language Inference (BART-MNLI), extracting distinctive group vocabulary via Fightin' Words Bayesian log-odds, and running OLS key-driver regression on star ratings or retention.
🎯 Questions It Answers
- To what degree (0% to 100%) does each customer review discuss custom operational constructs like Shipping Delay, Price Fairness, or Build Quality?
- Which specific words and phrases statistically distinguish 5-star vs. 1-star reviews or Brand A vs. Brand B?
- Which text construct has the largest dollar or star-rating impact when we regress our outcome on the construct probabilities?
📂 What Data You Need
- Review CSV/TSV (10+ Rows): One row per review, open-ended survey response, or support ticket.
- Text & Outcome Columns: A review text column plus a group label (for lexical contrast) and/or numeric outcome (e.g.,
star_rating). - Custom Construct Labels: 2 to 8 plain-English business constructs you want the Zero-Shot NLI model to score.
💡 What You'll Discover
- Zero-Shot Construct Matrix: Per-review entailment probabilities across all custom constructs (Multi-Label or Single-Label).
- Fightin' Words Volcano Plot: Dirichlet-smoothed log-odds z-scores highlighting statistically distinctive vocabulary.
- Key-Driver Regression & Clause Saliency: OLS regression coefficients linking text constructs to ratings, plus phrase-occlusion heatmaps.
🔍 Key Nuances: Zero-Shot NLI, Fightin' Words Shrinkage & Clause Occlusion
- How Zero-Shot NLI Scores Custom Constructs Without Training Data: Each review is treated as a Premise and paired with a natural-language Hypothesis (e.g.,
"This review discusses {label}."). The BART-MNLI transformer evaluates whether the review entails vs. contradicts the hypothesis. In Multi-Label (Independent) mode, a single review can score high on both Great Taste and Slow Delivery simultaneously. - Why "Fightin' Words" Beats Raw Word Ratios: Raw frequency ratios explode to infinity when a rare word appears twice in Group A and zero times in Group B. Monroe, Colaresi, and Quinn (2008) apply an Empirical Bayes Dirichlet prior proportional to background corpus frequency so only words with robust statistical evidence cross |z| ≥ 1.96.
- Auditing Neural Scores with Clause Occlusion: In the Review Inspector, masking one clause at a time measures how much the construct probability drops without that phrase, showing you the exact words driving the neural score.
📐 Under the Hood: Model Equations & Variable Definitions
Choose Your Data Source
Preset Case Studies
Curated real-world marketing review datasets ready to explore
Select a preset marketing case study above to load verified customer reviews, construct hypotheses, and managerial discussion prompts, or upload your own CSV on the right.
Upload Your Own Data
Bring a CSV or TSV with a review text column and group or rating columns
Drag & Drop review file (.csv, .tsv, .txt)
Include at least 10 rows with a text column (e.g. review_text) and a group or numeric outcome column (e.g. star_rating).
INPUTS & CONSTRUCT CONFIGURATION
🎛️ Guide to Model & Lexical Settings: What Each Control Means & Implications of Your Choices
-
Scoring Engine Mode:
Instant Contextual NLI + Golden BART Cache uses pre-computed
facebook/bart-large-mnlientailment probabilities for built-in cases and a fast deterministic contextual NLI scorer (handling negations, intensifiers, and contrast pivots) for custom constructs/uploads in <50 ms. Live Neural Transformer via Hugging Face CDN downloads a quantized ONNX NLI transformer (~68 MB) into your browser's local cache to run full neural self-attention directly on your device. - Review Text, Grouping & Quantitative Outcome (\(Y\)) Columns: The Review Text Column serves as the NLI Premise and the corpus for lexical tokenization. The Grouping Column partitions documents into discrete segments (e.g., store locations, firmware batches, skin types) for Chart 1 and Fightin' Words contrast. The Quantitative Outcome (\(Y\)) (e.g., 1–5 star rating, helpfulness votes, or \(0/1\) subscription renewal) is the dependent variable in the Key-Driver OLS Regression.
- Focal Group A vs. Reference Group B (Fightin' Words): Selects which two groups are contrasted in Section 4. Words with positive \(z_w > +1.96\) are statistically over-represented in Focal Group A; words with negative \(z_w < -1.96\) are statistically over-represented in Reference Group B.
-
NLI Hypothesis Template:
Controls how each construct label is framed as a sentence hypothesis for the NLI model:
"This customer review discusses {label}."— Topical framing: detects whether the topic is raised at all (positive or negative)."This customer review complains about {label}."— Negative/defect framing: isolates dissatisfaction and pain points."This customer review praises {label}."— Positive/delight framing: isolates brand strengths and value drivers."The reviewer experienced {label}."— Experiential/incident framing: best for concrete operational events (delays, side effects, hardware failures).
-
Construct Normalization Mode (Multi-Label vs. Single-Label):
- Multi-Label Independent (Recommended): Evaluates each construct independently via a binary entailment-vs-contradiction softmax \(P_k \in (0, 1)\). Because a single customer review often mentions both slow seating and cold food, Multi-Label preserves independent variation for Key-Driver OLS Regression.
- Single-Label Mutually Exclusive: Forces all \(K\) construct probabilities in a review to sum to \(1.0\) (\(\sum_{k=1}^K P_{ik} = 1\)). Analytical implication: Use only when classifying documents into mutually exclusive routing buckets; in OLS regression, summing to \(1.0\) induces structural multicollinearity (inflating \(\text{VIF}\)) because raising one construct's share mechanically depresses the others.
- Construct Presence Cutoff (\(\tau\)): Sets the probability threshold (default \(\tau = 0.50\)) used to classify a construct as "present" (\(\mathbb{I}(P_{ik} \ge \tau)\)) when computing Prevalence (\(\pi_k\)) in the Summary Table and Managerial Priority Matrix. Lowering \(\tau\) (e.g., to \(0.35\)) captures subtle or indirect mentions; raising \(\tau\) (e.g., to \(0.70\)) restricts prevalence counts to high-certainty mentions. Note that the OLS regression always uses the continuous probabilities \(P_{ik}\) rather than hard binary cutoffs.
- Dirichlet Prior Strength (\(\alpha_0\)): Controls Empirical Bayes shrinkage in the Monroe–Colaresi–Quinn Fightin' Words model (\(\alpha_w = \alpha_0 \cdot y_w / N\)). A low prior (\(\alpha_0 = 10\text{–}30\)) applies light smoothing, allowing lower-frequency niche terms to reach \(|z_w| \ge 1.96\). A high prior (\(\alpha_0 = 100\text{–}200\)) aggressively shrinks rare words toward zero so only high-volume, corpus-wide vocabulary differences survive.
-
Lexical Token Granularity & Minimum Corpus Word Frequency:
Unigrams Only contrasts individual words (fastest, cleanest baseline), whereas Unigrams + Bigrams includes two-word phrases (e.g.,
cold_food,battery_drain) that capture compound expressions. Minimum Corpus Word Frequency filters out typos and one-off hapax legomena before computing log-odds.
🎯 Target Business Constructs (\(K\) Candidate Hypotheses)
Define between 2 and 8 constructs to score across every review. Click × to remove a construct or add your own custom construct below.
💡 How to Design Effective Business Constructs & Avoid Multicollinearity (VIF)
-
Make Constructs Specific & Operational:
Zero-Shot NLI encoders extract much richer signal from concrete, domain-grounded phrases (e.g.,
Service Delay & Seating FrictionorBluetooth Multipoint Dropouts) than from generic single words likeServiceorQuality. Including 2–3 facet words in the label anchors the semantic neighborhood. -
Keep Constructs Conceptually Distinct (Watch Your VIF):
If you add two nearly synonymous constructs—such as
Wait Time & Seating DelayandSlow Hostess & Table Pacing—both will receive high entailment probabilities on the exact same reviews. In the Key-Driver OLS Regression, this collinearity inflates the Variance Inflation Factor (\(\text{VIF} = 1 / (1 - R_k^2)\)) above \(5.0\), widening standard errors and splitting the true regression slope across the duplicate constructs. - Include Both Pain Points and Value Drivers: To build an actionable Managerial Priority Matrix, include both hypothesized friction constructs (expected negative slope \(b_k < 0\)) and delight/strength constructs (expected positive slope \(b_k > 0\)) so you can compare how much star-rating lift positive experiences generate relative to the penalty of operational failures.
QUANTITATIVE CONSTRUCT & LEXICAL VISUALIZATIONS
1. Construct Entailment Profile by Group
Bars display mean Zero-Shot NLI entailment probability (\(\bar{P}_k\)) with 95% confidence intervals across each group in location.
Dashed horizontal line marks the active presence threshold.
💡 How to Read the Construct Entailment Profile
Each cluster compares how strongly different branches, cohorts, or customer segments express a specific construct.
When one bar towers above the others (for example, Harborplace on Service Delay & Seating Friction),
that operational issue is localized to that specific group rather than a brand-wide problem.
2. Construct Summary & Key-Driver OLS Regression on star_rating
Ordinary Least Squares multiple regression predicting outcome from all \(K\) construct entailment scores simultaneously.
| Construct Hypothesis | Mean \(P_k\) (SD) | Prevalence (\(\ge \tau\)) | Simple \(r\) | OLS Slope \(b_k\) | Std. \(\beta_k\) | SE | \(t\) | \(p\)-value | 95% CI for \(b_k\) | VIF |
|---|
3. Managerial Priority Matrix: Construct Prevalence vs. Outcome Impact (\(b_k\))
Horizontal axis shows Construct Prevalence (% of reviews scoring \(\ge \tau\)); vertical axis shows the partial OLS regression slope \(b_k\) on star_rating.
Constructs in the bottom-right quadrant are high-frequency value destroyers (urgent fixes); constructs in the top-right are signature strengths.
💡 How to Use the Managerial Priority Matrix
Managers often over-react to loud complaints that only occur in 3% of reviews, or ignore frequent complaints assuming "everyone complains about prices." Plotting Prevalence (\(\pi_k\)) against Regression Slope (\(b_k\)) isolates the constructs that are both widespread and statistically destructive to star ratings or retention.
4. Fightin' Words Lexical Contrast: Group A vs. Group B
Monroe, Colaresi, & Quinn (2008) Empirical Bayes Dirichlet-smoothed log-odds \(z\)-scores (\(z_w\)) plotted against log corpus frequency. Dashed horizontal cutoffs at \(z = \pm 1.96\) mark words that are statistically over-represented at \(\alpha = 0.05\).
| Term / Phrase | Count in Group A (per 1k) | Count in Group B (per 1k) | Smoothed Log-Odds \(\hat{\delta}_w\) | Posterior SE | Fightin' Words \(z_w\) | Distinctive Group |
|---|
5. Interactive Document & Clause Saliency Inspector (\(\Delta P_c\))
Click any review row on the left to inspect its full construct probability profile and leave-one-clause-out occlusion saliency shift \(\Delta P_c = P_{\text{full}} - P_{-c}\) on the right.
| ID | Group | Outcome | Focus \(P_k\) | Review Excerpt |
|---|
Clause Occlusion Attribution (\(\Delta P_c\))
Select a review on the left to inspect clause-level evidence.