Missing Data Lab
Your survey came back with 30% of income fields blank. Do you drop those rows? Fill them with the average? Use a smarter prediction? Each choice leads to a different reality in your downstream analysis — some close to truth, some dangerously wrong. This lab puts you in the driver's seat: inject missing data, choose your fix, and see what you present to your CMO.
OVERVIEW & WHAT THIS TOOL SOLVES
The real danger of missing data in marketing isn't that rows are blank—it's that the customers who leave blanks are rarely a random slice of your audience. High earners skip income fields, and dissatisfied buyers ignore follow-up surveys. When you blindly drop incomplete rows or plug in the column mean, you can distort regression slopes and segment averages by 30% to 50%. The Missing Data & Imputation Lab lets you inject controlled missingness (MCAR, MAR, MNAR) into known ground-truth data and compare how four treatment strategies perform.
🎯 Questions It Answers
- Why does it matter how values went missing—Completely at Random (MCAR), At Random given other columns (MAR), or Not at Random (MNAR)?
- How badly does Mean Imputation shrink variance and distort regression slopes compared to Conditional or Regression Imputation?
- How much bias (
% errorvs. true population parameters) survives under all four methods when missingness is MNAR?
📂 What Data You Need
- Scenario & Mechanism: Choose a preset marketing scenario encoding MCAR, MAR, or MNAR missingness
- Missingness Rate & Seed: Select the percentage of target values to blank out (10%–70%) and a reproducible RNG seed
- Imputation Strategy: Test Listwise Deletion, Mean/Mode Replacement, Conditional Group Imputation, or Regression Imputation
💡 What You'll Discover
- Missing vs. Observed Distribution Audit: Visual proof of whether missing rows come uniformly or from one tail of the distribution
- Ground-Truth Bias Reveal: Exact parameter distortion on the mean, SD, and downstream regression slope (
β̂ vs. β_true) - 4-Method Side-by-Side Scorecard: Direct comparison of all four imputation strategies against the true complete dataset
🔍 Key Nuances & Common Traps to Watch For
- MCAR vs. MAR vs. MNAR: Under MCAR (a random telemetry glitch), listwise deletion loses sample power but stays unbiased. Under MAR (missingness depends on an observed column like Tier), conditional/regression imputation recovers the truth. Under MNAR (people hide high income because it is high), every standard method retains bias.
- Why Mean Replacement Is Almost Always a Trap: Plugging
ȳ_obsinto 40% of rows creates an artificial spike at the center of the distribution, crushes standard deviations, and drags correlations toward zero. - Deterministic Regression Imputation Overstates Precision: Predicting missing values directly onto a regression line restores the slope better than mean replacement, but treats imputed guesses as noise-free facts unless random residual draws (stochastic/multiple imputation) are added.
📐 Under the Hood: Model Equations & Variable Definitions
1. Rubin's Missingness Mechanisms (MCAR, MAR, MNAR)
💬 Read it in plain English: Missingness R_i is MCAR if every row has the same chance of being blank, MAR if the chance depends only on other observed columns Y_obs, and MNAR if the chance depends on the unseen missing value Y_mis itself.
2. Imputation Estimators & Downstream Parameter Bias
💬 Read it in plain English: Mean imputation plugs in the overall observed average, conditional imputation plugs in the customer's segment average, regression imputation predicts the missing value from other features x_i, and Bias(%) measures how far the resulting downstream slope β̂_imp drifts from the true complete-data slope β_true.
STEP 1 — CHOOSE A SCENARIO
Each scenario has a distinct dataset, missingness mechanism, and downstream analysis. Start with Scenario 1 to calibrate; Scenarios 3 & 4 deliver the real surprises.
Email Survey Glitch
Missing: satisfaction_score
Outcome: repurchase probability
A server error corrupted survey responses at random. Some high scorers are missing, some low scorers — no pattern. The forgiving baseline.
Loyalty Tier Non-Response
Missing: email_open_rate
Outcome: conversion rate
Gold-tier customers ignore survey emails. Bronze customers always respond. Missingness is explained by a variable you do have.
Income Disclosure Trap
Missing: annual_income
Outcome: avg order value
High earners systematically refused to disclose income. The right tail of your distribution simply vanishes. Every method gets it wrong.
Silent Detractors
Missing: nps_category
Outcome: churn flag
Unhappy customers skip NPS surveys. Your collected data over-represents people who like you. Your NPS score will look fantastic — and be wrong.
STEP 2 — SET MISSINGNESS & INJECT
STEP 3 — EXAMINE THE WOUND
STEP 4 — CHOOSE YOUR TREATMENT METHOD
Select the strategy you believe will best recover the true relationship. You’ll see your downstream results before ground truth is revealed — so choose carefully.
Listwise Deletion
Drop every row where the focal column is blank. Use only complete cases.
Best when: Data is MCAR and missing rate is low (< 10%). Unbiased under MCAR but wasteful.
- ✅ Unbiased under MCAR
- ❌ Destroys rows and causes bias under MAR/MNAR
- ❌ Throws away all other columns for dropped rows
Mean / Mode Imputation
Fill missing values with the column average (continuous) or most frequent category.
Best when: Missing rate is very low (< 5%) and you just need a quick fix that won’t collapse variance too badly.
- ✅ Simple, preserves sample size
- ❌ Shrinks the column’s spread and R² — yet a mean-filled predictor keeps the listwise slope, so the slope won’t warn you
- ❌ Creates a spike at the mean in distributions
Conditional Imputation
Impute using within-group means or modes, conditioned on a related observed variable (e.g., loyalty tier, age group, churn flag).
Best when: Data is MAR and you know which variable explains the missingness pattern. Conditions on that variable to undo the systematic bias.
- ✅ Correct fix for MAR when conditioning variable is right
- ❌ Fails if the conditioning variable doesn't explain missingness
Predictive Imputation
Fit a regression (continuous) or probabilistic model (categorical) using all available complete rows; predict each missing value from other columns.
Best when: Other columns carry real predictive signal for the missing variable and the responders it learns from resemble the non-responders. Still biased under MNAR; whether it beats simpler fills depends on the data, so check it against the others in the Step 6 table.
- ✅ Keeps more of the column’s spread than one global mean when the auxiliary column predicts well
- ❌ Collapses to mean imputation when the auxiliary column predicts poorly
- ❌ Still biased under MNAR — the missing pattern is unobservable
STEP 5 — DOWNSTREAM IMPACT
Your Results (ground truth hidden)
💡 Why does this happen?
Under MCAR, missing values are a random sample. Any imputation strategy gets an unbiased estimate of the mean, and the slope stays close to the truth because nothing pushes it in a systematic direction. What imputation costs you is spread: filled-in values pile up at the centre, so the column's standard deviation and R² both fall.
Under MAR, the systematic pattern (e.g., Gold tier never responds) distorts the sample if you don't account for it. Conditional imputation essentially re-weights back to the correct group distributions — which is why it brings the mean open rate back close to the truth here. The slope is not where this damage shows: listwise deletion and mean imputation both leave it near the true value while the mean falls.
Under MNAR, the imputed value is always a guess about a biased sample. High earners' income is systematically absent from your data. Every imputation method plugs in a value from the observable lower part of the distribution — so the mean stays low and the high end of the scatter disappears no matter what. The slope does not collapse with it: because the gaps are in the predictor, it stays close to the truth (slightly steeper, not flatter, at the default seed), which is exactly what makes the damage easy to miss. The only real fix is to design the data collection process differently (e.g., bracket income ranges, verify through linked data, use Bayesian sensitivity analysis).
STEP 6 — ALL METHODS SIDE BY SIDE
Ground truth pinned as reference (the dark bar in the chart, the top row of the table). Mean Bias and β Bias give each method's percentage distance from it.
⚠️ What you'd report to a CMO: The table below is the most important thing to study. Every number that drifts from the ground-truth row (the mean and its Mean Bias, the β Bias, the R²) represents a question your CMO could ask (“What does our typical customer earn?”) that you'd answer wrongly because of how you handled missing data upstream.
📘 Key Takeaways
- MCAR: All four methods recover the truth reasonably. Listwise deletion loses rows (lower n) but doesn't introduce directional bias. Mean imputation leaves the slope exactly where listwise deletion put it, but it shrinks the column's spread and R²; it is acceptable at low % missing.
- MAR: Listwise deletion and mean imputation both drag the mean open rate well below the truth. Mean imputation does not flatten the slope (it reports the listwise slope exactly), but it collapses the within-tier spread and cuts R² sharply. Conditional imputation (using the correct conditioning variable) is the right tool — it brings the mean back close to the truth.
- MNAR (continuous): Every method is biased, and by almost the same amount: each one reports an average income well below the truth because each one learns only from the customers who answered. The slope stays close to the truth, so it will not warn you. The only real fix is better data collection.
- MNAR (categorical): Mode imputation amplifies the over-representation of Promoters. Listwise deletion removes most of the Detractors. The chi-square statistic weakens dramatically, hiding the churn-NPS link. Conditional imputation (using churn flag) can bring Cramér's V back near the truth while the category shares stay wrong; at the default settings it fills no one in as a Detractor. A recovered V does not mean recovered shares.
- Rule of thumb: At 10–15% MCAR, almost any approach is fine. Above 20% under MAR or any % under MNAR, your methodology section needs to explicitly justify your choice — or your results aren't trustworthy.