Rank-Based Tests
Comparison tool
Compare groups by rank, for when the average is not a safe summary: skewed spend, ratings, or one customer who is nothing like the others.
QUICK START: Choose Your Path
π What a rank test actually does
A rank test lines every case up from smallest to largest and works with each case's position rather than its value. The largest order in the file is simply last in line, whether it is twice the next one or fifty times it.
That is the whole trade. You give up the ability to say by how much, and in return no single case can drag the answer around.
DATA SOURCE
π
Use a Case Study
π
Use Your Data
One row per case, with a column naming the group and a column holding the number. The tool reads the shape of your file and picks the design to match.
Drag & Drop your file (.csv, .tsv, .txt)
Headers like group,value. At least 6 rows.
One column per group also works and is detected automatically, so a spreadsheet laid out that way needs no rearranging. There is no template for it because long format is the shape worth learning: every other tool here reads it.
Messy file? The Import Wizard will check it and tell you what to fix. Files above about 5 MB are slow in the browser; take a sample of the rows instead.
No file uploaded.
One box per group. Paste a column from a spreadsheet, or type the numbers separated by spaces, commas or new lines. Good for the ten numbers on a whiteboard.
Nothing entered yet.
INPUTS & SETTINGS
What kind of comparison is this?
Pick by how the data was collected, not by the name of a test. The test follows from the answer.
Units on the values
Confidence level
Advanced
By default a pair that did not change is set aside before ranking, which is Wilcoxon's own method. Pratt's keeps them and gives a different p-value, so whichever you use has to be reported.
Nothing to set for this design.
Order of the groups
Drag to reorder, or use Alt and the arrow keys. This changes how the groups are listed and charted. It cannot change the test result.
How often would a gap this big show up by chance?
–p-value
Load a case to see what this means here.
How often is the higher group actually higher?
–Probability of superiority
What did the test itself come out at?
–Mann-Whitney U
Report it alongside the p-value, not instead of it.
Those three names are the ones you will be graded on. The questions above them are what each one is actually answering.
Start here
Group comparison & rank distributions
Each group
| Group | n | Median | Middle half | Mean rank | Mean |
|---|
Managerial Interpretation
Diagnostics & assumptions
What this test is assuming about your data
Nothing here needs your attention. A rank test asks less of the data than a t-test does, which is most of why you would reach for one.
Which pairs actually differ (Dunn's test)
| Comparison | Mean rank gap (ΞRΜ) | Dunn z | Unadjusted p | Adjusted p | How often the higher one wins | Verdict |
|---|
Six comparisons on four groups means six chances to find something by accident, so
Holm's step-down rule multiplies each unadjusted p-value by the number of remaining tests (capped at 1.000). When two groups have nearly identical mean ranks (e.g., Paid social, Search, and Affiliate all sit around rank 138β144 while Referral sits at rank 57), their unadjusted p-values are already 0.61β0.83, which clamp to 1.000 after adjustment.
What the t-test says about the same data
Compare the two tests
| Test | p-value | Which group comes out ahead |
|---|---|---|
| Welch's t-test (compares averages) | ||
| Mann-Whitney (compares ranks) |
Nothing is deleted and nothing is downloaded. Both tests above re-run without that one case so you can see which of them it was moving.
OVERVIEW & WHAT THIS TOOL SOLVES
A single "whale" enterprise order or a handful of extreme B2B invoices can pull a group's average so hard that a standard t-test or ANOVA points in the wrong directionβor misses a genuine shift across typical customers. Rank-Based (Nonparametric) Tests line every observation up from smallest to largest and analyze each case's sorted position rather than its raw dollar value, protecting your comparison against extreme skewness, heavy tails, and ordinal survey scales.
π― Questions It Answers
- 2 Independent Groups (Mann-Whitney U): Does one group consistently outrank the other when orders or engagement are heavily skewed?
- Paired Before/After (Wilcoxon Signed-Rank W): Did typical within-customer scores shift when change scores have extreme outliers?
- 3+ Groups (Kruskal-Wallis H + Dunn's Test): Which channels or segments differ significantly in rank distribution?
π What Data You Need
- Case-Level CSV or Manual Entry: At least 6 observations across two independent groups, paired before/after columns, or 3+ groups.
- Numeric or Ordinal Outcome: Skewed dollar metrics, count data, time-on-site, or 1β5 / 1β7 ordinal rating scales.
- Auto-Design Detection: The tool inspects your columns and selects Mann-Whitney U, Wilcoxon W, or Kruskal-Wallis H automatically.
π‘ What You'll Discover
- Rank Test vs. Parametric Comparison: Side-by-side comparison showing whether a single extreme outlier is flipping the t-test verdict.
- Intuitive Effect Sizes: Probability of Superiority (PS), Rank-Biserial Correlation (rrb), and Epsilon-Squared (Ξ΅2).
- Dunn's Post-Hoc Pairwise Table: Tie-corrected pairwise rank comparisons with Holm step-down adjusted p-values for 3+ groups.
π Key Nuances: Choosing Rank vs. Parametric Tests & Effect Size Benchmarks
- Two Separate Groups (Mann-Whitney U vs. Welch's t-test): Use Mann-Whitney U when order value, spend, or time-on-site has strong right-skew or whale outliers, or when the outcome is an ordinal rating scale.
- Same People Twice (Wilcoxon Signed-Rank W vs. Paired t-test): Ranks the absolute magnitudes of non-zero within-person changes and compares positive vs. negative rank sums.
- Three or More Groups (Kruskal-Wallis H vs. One-Way ANOVA): Tests whether at least one group's rank distribution stands apart, followed by Dunn's post-hoc test to pinpoint which specific pairs differ.
- Effect Size Benchmarks: Probability of Superiority (PS) = 0.50 is a coin flip, 0.56 is small, 0.64 is moderate, and 0.71+ is strong; Rank-Biserial Correlation (rrb) ranges from β1 to +1 (Β±0.10 small, Β±0.30 moderate, Β±0.50 large); Epsilon-Squared (Ρ²) measures the share of rank variance explained by group membership (0 to 1).
π Under the Hood: Model Equations & Variable Definitions
1. Two Independent Groups: Mann-Whitney U, Probability of Superiority & Rank-Biserial
\[ U_1 = R_1 - \frac{n_1(n_1 + 1)}{2}, \qquad U_2 = n_1 n_2 - U_1, \qquad PS = \frac{U_1}{n_1 n_2}, \qquad r_{rb} = 2 \cdot PS - 1 \]
π¬ Read it in plain English:
Mann-Whitney U counts how many times an observation in Group 1 outranks an observation in Group 2 across all n1 Γ n2 pairs; dividing by total pairs gives the Probability of Superiority (PS).
R1, n1, n2
Pooled Rank Sum & Group Sample Sizes
YOUR DATA β
Sum of pooled ranks in Group 1 (R1) and sample sizes in Group 1 and Group 2 (n1, n2).
U1, U2, PS, rrb
Mann-Whitney U & Superiority Effect Sizes
INFERENCE β
Pairwise rank win counts (U1, U2), Probability of Superiority (PS), and Rank-Biserial net win share (rrb).
2. Paired Data (Wilcoxon W) & 3+ Groups (Kruskal-Wallis H & Epsilon-Squared)
\[ W^+ = \sum_{\Delta_i > 0} \mathrm{rank}(|\Delta_i|), \qquad W^- = \sum_{\Delta_i < 0} \mathrm{rank}(|\Delta_i|), \qquad r_{rb,\text{paired}} = \frac{W^+ - W^-}{W^+ + W^-} \]
\[ H = \frac{\frac{12}{N(N + 1)} \sum_{j=1}^{k} \frac{R_j^2}{n_j} - 3(N + 1)}{C_{\text{ties}}}, \qquad \varepsilon^2 = \frac{H}{N - 1} \]
π¬ Read it in plain English:
For paired data, Wilcoxon compares the rank sum of positive changes (W+) against negative changes (Wβ); for 3+ groups, Kruskal-Wallis H measures how far each group's average rank diverges from the overall average rank across all N cases.
Ξi, W+, Wβ
Paired Difference & Signed Rank Sums
OUTPUT β
Within-pair change score (Ξi) and sums of absolute-change ranks for positive (W+) and negative (Wβ) shifts.
H, Rj, Cties, Ξ΅2
Kruskal-Wallis H, Tie Correction & Epsilon-Squared
INFERENCE β
Omnibus rank statistic H across k groups (using group rank sums Rj and tie correction Cties) and effect size Ξ΅2.