Propensity Score Matching

Causal Inference

Estimate causal treatment effects from observational marketing data using propensity score matching. Upload data with a treatment indicator and outcome, select covariates for balance, and the tool performs nearest-neighbor or caliper (radius) matching to create comparable treated and control groups.

1 ✓
Upload Data
2 ✓
Configure Variables
3 ✓
Run Matching
4 ✓
Interpret Results

👨‍🏫 Professor Mode: Guided Learning Experience

New to propensity score matching? Enable Professor Mode for step-by-step guidance through matching methods, balance diagnostics, and interpreting causal treatment effects!

OVERVIEW & WHAT THIS TOOL SOLVES

Enrolled customers self-select — they look systematically different from non-enrolled. Propensity scores compress all covariates into one matchable number. Each treated customer finds its nearest statistical twin. Compare only within pairs — the gap is the causal effect, not selection bias.

A company launches a loyalty program: enrolled members spend $1,840/year while non-members spend $1,210—does the program truly drive $630 in incremental spend? Almost certainly not: your best, highest-spending customers self-selected into enrolling and would have spent more anyway. Propensity Score Matching (PSM) pairs each treated customer with a non-enrolled "statistical twin" who looked equally likely to enroll beforehand, stripping away selection bias to isolate the true causal lift.

🎯 Questions It Answers

  • How much of the raw outcome gap is true causal program lift versus pre-existing customer self-selection?
  • Did our loyalty program, CRM campaign, or sales call cause higher spend among the customers it reached?
  • Are our treated and control groups genuinely comparable on pre-treatment characteristics after matching?

📂 What Data You Need

  • Observational customer or account records (one row per unit).
  • A binary treatment column (0/1) (e.g., enrolled vs. not) and a numeric outcome column.
  • Pre-treatment covariates that predict who self-selects into treatment (prior spend, tenure, visits).

💡 What You'll Discover

  • Causal Effect on the Treated (ATT): Matched-pair lift compared side-by-side against Naive and OLS estimates.
  • Covariate Balance Diagnostics: Standardized Mean Differences (SMD) before and after matching (Love plot).
  • Common Support & Overlap: Propensity score distributions showing matched vs. unmatched units.
🔍 Key Nuances, Common Traps & Organizational Reality Check Click to expand ▾
  • Selection Bias Decomposition: Every naive observational gap equals Real Program Effect + Selection Artifact. Because loyal, high-value customers self-select into marketing programs, PSM will almost always produce a smaller, more honest effect estimate than the naive comparison.
  • Observed vs. Unobserved Confounders: PSM balances the covariates you actually measured and included. If an unmeasured factor (e.g., unrecorded brand enthusiasm) drove both enrollment and spending, hidden bias can remain.
  • ATT vs. ATE: 1:1 matching estimates the Average Treatment Effect on the Treated (ATT)—the causal lift among the customers who actually enrolled, not necessarily what would happen if you forced enrollment on the entire population (ATE).
  • Organizational Reality Check: Because PSM strips away selection inflation, stakeholders sometimes resist the smaller matched number or pressure analysts to widen calipers or drop covariates. Defending a smaller, credible causal estimate is one of the highest-value contributions an analyst can make.
📐 Under the Hood: Model Equations & Variable Definitions Click to expand ▾
1. Step 1 — Propensity Score Estimation (Logistic Selection Model) Balancing Score
\[ e(X_i) = \Pr(T_i = 1 \mid X_i) = \frac{1}{1 + \exp\bigl(-(\beta_0 + \beta_1 X_{1i} + \cdots + \beta_p X_{pi})\bigr)} \]
💬 Read it in plain English: Compresses all of customer \(i\)'s pre-treatment characteristics \(X_i\) into a single probability between 0 and 1—how likely someone with that exact profile was to self-select into treatment.
Variable & Symbol Directory (Every Term Defined) Hover or focus any role badge for details
e(Xi)
Propensity Score OUTPUT ⓘ
Predicted probability (\(0\) to \(1\)) that customer \(i\) enrolls based on their pre-treatment profile.
Ti
Treatment Indicator (0/1) YOUR DATA ⓘ
Equals \(1\) if customer \(i\) enrolled / received treatment, and \(0\) if in the untreated control pool.
X1i,…,Xpi
Pre-Treatment Covariates YOUR DATA ⓘ
Observed baseline characteristics of customer \(i\) (e.g., prior spend, visit frequency, tenure) measured before treatment.
β0,…,βp
Selection Model Log-Odds Weights ESTIMATED ⓘ
Logistic regression coefficients showing how strongly each baseline covariate predicts self-selection into treatment.
2. Steps 2 & 3 — Matched-Pair Average Treatment Effect on the Treated (ATT) Causal Effect Estimator
\[ \widehat{\text{ATT}} = \frac{1}{n_1} \sum_{i:\, T_i = 1} \bigl( Y_i - Y_{j(i)} \bigr) \]
💬 Read it in plain English: For every treated customer \(i\), subtract the outcome of their closest propensity-score control twin \(j(i)\), and average those within-pair differences across all \(n_1\) matched pairs.
Variable & Symbol Directory (Every Term Defined) Hover or focus any role badge for details
ATT
Average Treatment Effect on Treated OUTPUT ⓘ
Mean causal outcome gain among treated customers compared to their matched statistical twins.
n1
Matched Treated Count YOUR DATA ⓘ
Total number of treated customers (\(T_i = 1\)) with a valid matched control twin inside the caliper.
Yi
Treated Customer Outcome YOUR DATA ⓘ
Observed outcome (e.g., annual spend) for treated customer \(i\).
Yj(i)
Matched Control Twin Outcome YOUR DATA ⓘ
Observed outcome for the untreated control customer \(j(i)\) paired as the nearest neighbor to treated customer \(i\).

DATA SOURCE

📚

Use a Case Study

Use presets to explore causal inference scenarios, such as loyalty program enrollment or email campaign exposure. Each scenario provides a raw data file with treatment, outcome, and covariates.

📤

Upload Your Data

Upload a CSV file with case-level data including: a binary treatment indicator (0/1), an outcome variable, and covariates for matching. Headers are required.

Drag & Drop raw data file (.csv, .tsv, .txt, .xls, .xlsx)

Include headers; must have treatment (binary), outcome, and covariate columns.

No file uploaded.

DATA & VARIABLE SELECTION

PROPENSITY SCORE DIAGNOSTICS

Propensity Score Distribution

Interpretation Aid

Technical Interpretation

What you're seeing: The histograms show the distribution of estimated propensity scores for treated (blue) and control (orange) groups. Each propensity score represents the predicted probability that a unit received treatment, based on its covariate values.

Good overlap: When both distributions cover similar ranges (e.g., both spanning 0.2–0.8), matching can find comparable units. The key assumption of PSM—that we can find similar treated and control units—is satisfied.

Warning signs: If treated units cluster near 1.0 and controls near 0.0 with little overlap, the propensity model is "perfectly predicting" treatment. This means groups are fundamentally different on observed characteristics, and matching will fail or produce few matches.

Practical Interpretation

Marketing example: In a loyalty program analysis, if customers who enrolled all have very high propensity scores while non-enrollees have very low scores, it means enrolled customers were systematically different (e.g., higher prior spend, more visits). Finding a "fair comparison" will be difficult.

What to do if overlap is poor: (1) Use fewer or different covariates, (2) collect more data, or (3) consider that the treatment effect may not be estimable with this data—the groups are too different to compare.

Covariate Balance: Before vs. After Matching

Interpretation Aid

Technical Interpretation

Standardized Mean Difference (SMD): Each point shows how different treated and control groups are on a covariate, measured in pooled standard deviation units. An SMD of 0.2 means the groups differ by 0.2 standard deviations on that variable.

Balance thresholds: SMD < 0.1 indicates excellent balance (green zone). SMD 0.1–0.25 is acceptable but shows residual imbalance (yellow). SMD > 0.25 suggests meaningful differences that may bias the ATT estimate (red).

Before vs. After: Open circles show pre-matching imbalance; filled circles show post-matching balance. Successful matching moves all points toward zero (the center dashed line).

Practical Interpretation

What it tells you: This "Love Plot" shows whether matching created comparable groups. If all filled circles (after matching) are within the ±0.1 bands, you can be confident that treated and matched control groups are similar on observed characteristics.

Marketing example: If "Prior_Spend" has SMD = 0.5 before matching (treated spent more) but SMD = 0.05 after matching, the matched comparison controls for prior spending behavior. The ATT now compares customers with similar spending histories.

Action: If any covariate remains imbalanced (SMD > 0.25) after matching, consider adding it to the outcome model or interpreting results cautiously—remaining confounding may bias the effect estimate.

MATCHING SUMMARY

Treated Units --
Control Units --
Matched Pairs --
Unmatched Treated --
Mean PS (Treated) --
Mean PS (Matched Control) --
Understanding Matching Quality

Technical Interpretation

Matched pairs: The number of treated units successfully matched to control units within the caliper threshold. This determines the effective sample size for ATT estimation.

Unmatched treated: Treated units outside the "common support" region—they have propensity scores too extreme to find comparable controls. High unmatched rates (>20%) suggest limited overlap.

Mean propensity scores: After matching, mean PS should be nearly identical between treated and matched controls. Differences > 0.05 suggest residual imbalance.

Practical Interpretation

Sample size trade-off: PSM often discards observations (unmatched treated or unused controls). A match rate of 80%+ is typically good; below 50% suggests the groups may be too different for meaningful comparison.

Who gets excluded: Unmatched treated units are often "extreme" cases—e.g., customers with very high engagement who enrolled in a program but have no comparable non-enrollees. The ATT estimate applies only to the matched subset, which may differ from all treated units.

Generalizability: If many treated units are excluded, ask: "Can I generalize the treatment effect to the excluded cases?" If they're systematically different, the answer may be no.

COVARIATE BALANCE TABLE

Standardized Mean Differences

Covariate Mean (Treated) Mean (Control) SMD Before Mean (Matched Trt) Mean (Matched Ctrl) SMD After % Reduction
Run matching to see balance diagnostics.

SMD interpretation: |SMD| < 0.1 = excellent balance (green), 0.1–0.25 = acceptable (yellow), > 0.25 = concerning (red).

About Standardized Mean Differences

Technical Interpretation

What SMD measures: The difference in means between treated and control groups, divided by the pooled standard deviation: SMD = (Mean_Treated - Mean_Control) / SD_pooled. This makes the measure unit-free and comparable across variables.

For categorical variables: Each category level is treated as a 0/1 indicator, and SMD is calculated for the proportion in each level.

% Reduction: Shows how much matching improved balance. 80%+ reduction is excellent; negative reduction means matching made things worse (rare, but possible with replacement).

Practical Interpretation

Reading the table: Compare the "SMD Before" and "SMD After" columns. Ideally, all "SMD After" values should be near zero (green). Covariates with high SMD after matching may still confound your treatment effect estimate.

Example: If "Prior_Spend" has SMD = 0.4 before matching, treated customers spent 0.4 SDs more than controls before the program. After matching (SMD = 0.05), the difference is negligible—you're now comparing customers with similar spending histories.

If balance is poor: Try adjusting matching settings (wider caliper, different covariates) or consider that selection bias may be too strong to overcome with this data.

TREATMENT EFFECT RESULTS

Average Treatment Effect on the Treated (ATT)

Estimated ATT: --
Standard Error: --
95% CI: --
t-statistic: --
p-value: --
Mean Outcome (Treated): --
Mean Outcome (Matched Control): --
Interpreting the ATT

Technical Interpretation

Average Treatment Effect on the Treated (ATT): The mean difference in outcomes between treated units and their matched controls: ATT = (1/n) Σ(Y_treated - Y_matched_control). This estimates the causal effect of treatment for those who received it.

Standard Error & 95% CI: The SE measures uncertainty in the ATT estimate. The 95% CI means we are 95% confident the true effect lies within this range. If the CI excludes zero, the effect is statistically significant at α = 0.05.

t-statistic & p-value: The t-stat is ATT/SE; p-value tests H₀: ATT = 0. p < 0.05 provides evidence of a non-zero treatment effect.

Practical Interpretation

What the number means: If ATT = 120 for a spending outcome, treated customers spent an average of $120 more than similar untreated customers. This is the estimated causal impact of the treatment.

Business decision-making: Compare the ATT to the cost of treatment. If a loyalty program costs $50/customer to run but generates ATT = $120 in additional spend, the program has positive ROI.

Caution: The ATT is valid only if (1) the propensity model includes all confounders, (2) matching achieved good balance, and (3) there are no unmeasured confounders. PSM cannot prove causation—it can only adjust for observed differences.

Effect size vs. significance: A large p-value with a meaningful ATT suggests insufficient sample size. A tiny p-value with a tiny ATT may be statistically significant but practically irrelevant. Always consider both.

Statistical Interpretation

Run matching analysis to see results.

Managerial Takeaway

Run matching analysis to see actionable insights.

The downloaded file includes the matched sample with propensity scores, match IDs, and outcomes.

METHODS COMPARISON: WHAT DID MATCHING BUY US?

Propensity score matching is most compelling when compared against simpler alternatives. The chart below shows three estimates of the treatment effect from the same data — revealing how much selection bias the naive comparison contains, and whether parametric OLS adjustment or PSM better handles it.

Understanding the Method Comparison

Why Three Methods?

Each approach makes different assumptions about selection bias and uses the data differently. Comparing them exposes how much of the raw effect was real vs. pre-existing group differences.

Naive Mean Difference: Simply compares treated vs. control group averages with no adjustment. Almost always the largest estimate — it conflates the treatment effect with pre-existing differences between who self-selected into treatment.

OLS with Covariates: Fits a linear regression with treatment + covariates as predictors. Adjusts for confounders parametrically (assumes linearity) and uses the full sample — even control units far outside the treated group's range. A sensible baseline but can extrapolate in ways that are hard to audit.

PSM ATT: Restricts the comparison to matched pairs only, enforcing the common support assumption. More transparent about which comparisons are being made. It explicitly discards unmatched units rather than extrapolating over them — at the cost of a smaller effective sample.

Reading the Forest Plot

Each row shows one method's point estimate (diamond) with 95% confidence interval (horizontal whiskers). The dashed vertical line at zero is the null (no effect). A CI that crosses zero means that method's estimate is not statistically significant at α = .05.

Interpreting the Pattern

If all three are similar: Selection bias was modest — the raw and adjusted estimates agree. Treatment effect is robust to method choice.

If PSM is much lower than naive: Selection bias was large. Treated units had favorable characteristics before treatment; the raw gap overstated the causal effect.

If OLS ≈ PSM but both < naive: The confounders you included capture most of the selection mechanism; both adjustment approaches agree.

If OLS and PSM diverge substantially: The linear OLS model may be misspecified, or treatment effects are heterogeneous across the propensity score distribution. PSM, by restricting to common support, is generally more defensible in this case.

PROPENSITY MODEL DETAILS

View Propensity Score Model Coefficients

The propensity score model predicts treatment assignment from covariates using logistic regression.

Run matching to see the propensity model.
Covariate Coefficient Std. Error z p-value Odds Ratio
Run matching to see coefficients.
Pseudo R²: -- Model χ²: -- p-value: --