Login Create Account

Bandit vs. A/B Race

Experimentation Simulation Lab

You have three email subject lines and one send. To find out which works best you have to send some people the weaker ones — and every one of those is a sale you probably gave up. This lab lets you watch three different ways of handling that trade-off compete for the same customers, and shows you exactly what each one costs.

Never heard of a “bandit” before? Turn on Professor Mode and it will walk you through one race from start to finish — what the chart is showing, why the winner wins, and the catch that nobody mentions until it is too late.

OVERVIEW & WHAT THIS TOOL SOLVES

Traditional A/B tests split traffic 50/50 until the test ends—meaning if Variant B is much better, you deliberately send half your shoppers to the losing creative for weeks and pay for statistical certainty in lost conversions. Multi-Armed Bandits dynamically shift traffic toward winning arms as evidence accumulates, trading off statistical power on losing arms in exchange for higher cumulative conversions and lower opportunity cost during the run.

🎯 Questions It Answers

  • How many conversions (and dollars) do we lose during an experiment by locking traffic in a fixed 50/50 split?
  • How do Thompson Sampling, UCB1, and ε-Greedy compare against a classical A/B test on single runs and across 500 Monte Carlo races?
  • When does a bandit's adaptive allocation hurt your ability to report a statistically significant p-value or handle a mid-campaign shift?

📂 What Data You Need

  • True Arm Conversion Rates: Preset marketing scenarios or custom conversion rates for 2 to 4 creative variants
  • Traffic Budget & Batching: Total visitors (T), decision update batch size, and order value
  • Monte Carlo Replications: Number of repeated simulated races to expose run-to-run luck

💡 What You'll Discover

  • Cumulative Regret & Reward Curves: Visitor-by-visitor tracking of missed conversions vs. an omniscient oracle
  • Dynamic Traffic Allocation: How quickly each algorithm starves losing variants and concentrates on the leader
  • Inference vs. Earning Trade-Off: Winner-pick accuracy alongside statistical power and confidence interval width
🔍 Key Nuances & Common Traps to Watch For

What is Cumulative Regret? If the best banner converts at 6% and a weaker banner converts at 2%, sending 1,000 shoppers to the weaker banner yields ~20 orders instead of ~60. Those 40 missing orders are called regret—the conversions you gave up compared to an omniscient oracle that knew the best arm from visitor #1.

  • Use a Bandit for Short-Lived Earning: Flash sales, holiday hero banners, and headline rotations where maximizing in-flight conversions matters more than publishing a precise lift estimate afterward.
  • Use a Fixed A/B Test for Permanent Strategic Decisions: Checkout redesigns, pricing changes, or architecture shifts where you need unbiased confidence intervals and high statistical power to prove how much better the winner is.
  • Non-Stationary Drift Can Fool Early Lock-In: If customer preferences shift mid-campaign, a bandit that starved Arm B in week 1 may take too long to notice Arm B surged in week 2.
📐 Under the Hood: Model Equations & Variable Definitions

1. Cumulative Regret & Upper Confidence Bound (UCB1)

\[ R(T) = \sum_{t=1}^{T} \left(\mu^* - \mu_{a_t}\right), \qquad \text{UCB1}_k(t) = \hat{\mu}_k(t) + \sqrt{\frac{2 \ln t}{n_k(t)}} \]

💬 Read it in plain English: Cumulative regret sums the expected conversion gap between the best possible arm (μ*) and the arm actually shown to each visitor t, while UCB1 picks the arm with the highest "optimistic" rate (observed conversion rate plus an uncertainty bonus for under-tested arms).

T, t
Total Traffic & Visitor StepYOUR DATA ⓘ
Total experiment horizon T and current visitor index t = 1, …, T.
μk, μ*
Arm Rate & Oracle Best RateYOUR DATA ⓘ
True conversion probability of arm k and the maximum rate μ* = max_k μ_k.
R(T)
Cumulative Expected RegretOUTPUT ⓘ
Expected conversions lost over T visitors compared to always playing the best arm.
μ̂k(t), nk(t)
Empirical Arm Mean & PullsESTIMATED ⓘ
Observed conversion rate and number of visitors allocated to arm k up to step t.
UCB1k(t)
Upper Confidence IndexINFERENCE ⓘ
Optimistic score balancing exploitation (μ̂_k) and exploration bonus √(2 ln t / n_k).

2. Thompson Sampling (Beta-Bernoulli Posterior) & ε-Greedy

\[ \theta_k(t) \sim \text{Beta}\!\left(1 + S_k(t),\; 1 + F_k(t)\right), \qquad P(\text{play } k \text{ under } \varepsilon\text{-Greedy}) = (1 - \varepsilon)\,\mathbb{I}(k = k^*) + \frac{\varepsilon}{K} \]

💬 Read it in plain English: Thompson Sampling maintains a Beta probability distribution over each arm's conversion rate using its successes S_k and failures F_k, drawing a random sample from each arm to route traffic in proportion to the probability that an arm is the true best.

Sk(t), Fk(t)
Arm Successes & FailuresYOUR DATA ⓘ
Cumulative conversions S_k and non-conversions F_k observed on arm k by step t.
θk(t)
Posterior Thompson DrawESTIMATED ⓘ
Random draw from arm k’s Beta(1 + S_k, 1 + F_k) posterior belief distribution.
ε, K
Exploration Rate & Arm CountYOUR DATA ⓘ
Fraction of traffic ε routed uniformly at random across all K competing variants.

Step 1 PICK A SITUATION

Each of these is a real marketing decision with real money attached. Start with Nordvik Travel if this is your first time — it is the simplest version, just two homepage images and a clear winner. The others get more interesting: one where the options are nearly identical, one where the world changes halfway through, and one where there is genuinely nothing to find.

Step 2 SET UP THE RACE

Because this is a simulation, you get to play god: you set the true conversion rate of each option, and then watch whether the three approaches can figure out what you already know. In a real test you would never have this information — that is the entire reason testing exists. If a preset is loaded, these are already filled in and you can go straight to Step 3.

The options

Set the true conversion rate of each option. In a real test you would not know these — that is the point of testing.

Option True conversion rate

With this on, the rates above are masked until the race ends, so you face the same uncertainty the algorithms do.

The race

Contestants

Three different answers to the same question: “where should I send the next visitor?” Leave all three ticked so you can compare them.

%

At 10%, one visitor in ten is sent to a random option regardless of how badly it has been doing. That is the price epsilon-greedy pays to avoid getting stuck on a bad early guess.

Advanced: let the world change mid-test

Step 3 RUN IT

Run one race sends every visitor through, one at a time, and shows you what happened on this particular occasion. Repeat the race does the whole thing over and over with different luck each time, which is the only way to tell a real pattern from a fluke.

More repeats give a steadier answer and take longer. You can stop a run at any point and nothing is lost.

🤔 Try this:

  1. Load Nordvik Travel and click Run one race. Look at the Scoreboard. Thompson sampling almost certainly earned the most conversions. Case closed?
  2. Now look further down at What each contestant can tell you at the end. Find the row for the option that lost. Compare how many visitors the A/B test sent there against how many Thompson sent, and look at how much wider Thompson's interval is.
  3. That width is the catch. Thompson made more money and learned less. If your boss asks “how much worse was the old banner?”, the A/B test can answer and Thompson has to shrug.

EXPLORATORY QUESTIONS

Each of these is answerable with the controls on this page. They are in rough order of difficulty, and the last two are the ones worth arguing about.

  1. How expensive is patience? Run Nordvik Travel. Note the A/B test's regret, then note Thompson's. The difference is what the A/B test paid for a defensible answer. Would you pay it?
  2. Does more exploring help? Drag ε from 10% up to 40% and re-run. Epsilon-greedy now explores four times as hard. Does its regret go up or down, and does it get better at picking the right option?
  3. What if there is nothing to find? Load Arcadia Software, where all four pricing pages convert identically. Everyone's regret is zero — so has everyone done equally well? Look at what each one would report.
  4. Can a bandit be fooled? Load Lumen Skincare, where the three subject lines are only a few tenths of a percentage point apart. Repeat the race 250 times and look at “Picked the right one”. Would you trust any of these to choose your Black Friday subject line?
  5. What happens when the world moves? Load Kettle & Co., where the winning banner changes in December. Which approach notices? Before you run it, write down your prediction — most people guess wrong.
  6. Why does exploring not rescue it? In that same scenario, push ε to 30% so epsilon-greedy is constantly sampling everything. It still gets the answer wrong. What is actually broken — is it not looking enough, or is it the way it is averaging what it sees?