Determine if your A/B test shows a real winner or just random noise—with instant decision guidance, business impact calculations, and visual evidence.
Select a preset scenario to explore real-world A/B testing situations with authentic marketing metrics and business context.
Required columns: group (two labels) and conversion (0 or 1).
Drag & Drop raw data file (.csv, .tsv, .txt, .xls, .xlsx)
Two columns: group label and conversion (0 or 1). Up to 5,000 rows.
Slide values or type directly into number boxes. Every result reads the number boxes, so a typed value such as 0.036 or 16342 is used exactly. Have counts instead of a rate? Type the conversions: 147 out of 1,250 sets the rate to 0.1176 exactly. The box you typed in last is the one held when you change the sample size.
Apply FPC when population size is known or approximately known. Used for sampling without replacement from a finite population. FPC reduces standard error when sample size is >5% of total population.
The share your test plan assigned to this group (50 for an even split). The diagnostics check the observed arm sizes against it.
We'll calculate significance and show you what to do next
When to use this calculator:
When NOT to use this calculator:
Many valuable A/B tests don't have explicit dollar values attached—that's perfectly fine! Skip this section if your test measures engagement, awareness, or other non-revenue metrics.
The fan charts show the uncertainty around your estimates. Narrower fans = more precision.
⚠️ Small Sample Warning
| Measure | Label | Proportion / Δ | CI Lower | CI Upper | n |
|---|
When Variant B converts at 4.8% vs. Control A's 4.2%, declaring victory on raw percentages alone is a classic marketing trap: random traffic luck regularly produces 0.6-point swings in small samples. A/B Proportion Testing evaluates whether the conversion gap between two independent groups exceeds sampling noise, pairing exact pooled/unpooled z-tests with confidence intervals and an optional dollar ROI calculator.
Δ = P₂ − P₁)?x₁, x₂) or proportions (P₁, P₂) and sample sizes (n₁, n₂) for Control and TreatmentΔ₀, finite-population correction (FPC), and business ROI inputsΔ₀ = 0 (or unpooled when testing against a non-zero hurdle Δ₀)P̄, so pooling gives the exact score test (where z² = χ²). For the confidence interval—or when testing a non-zero benchmark Δ₀ ≠ 0—the two rates differ, so the standard error remains unpooled.p < 0.05 inflates your false-positive rate from 5% to 20%+. Commit to your sample size upfront using the Sample Size A/B Calculator.💬 Read it in plain English: When testing whether two groups have the exact same true conversion rate, we pool their total conversions over total visitors to get P̄, then divide the observed rate gap (P₂ − P₁) by the pooled standard error.
💬 Read it in plain English: For confidence intervals and non-zero hurdle tests (Δ₀ ≠ 0), each group keeps its own variance P_i(1 − P_i)/n_i so the uncertainty interval reflects the actual conversion rates observed in each arm.
Diagnostics will appear after you enter data.