Design, run, and interpret randomized experiments — the marketer's most powerful causal tool. Master proportion tests, multi-variant designs, sequential testing, multi-armed bandits, and Bayesian methods used by the world's leading experimentation platforms.
Topics and managerial frameworks covered in A/B Testing & Marketing Experiments:
Students manipulate these interactive visual simulations directly inside the chapter reading before answering gated check-in questions:
Run an A/B test with interim looks and adjusted thresholds, stopping early for a clear win or for futility instead of waiting out a fixed end date.
Watch traffic shift automatically toward variants that are performing well while uncertain variants keep being explored, and compare the result against a standard A/B test.
This chapter connects directly to these full MktPraxis analytical apps (with pre-loaded business case scenarios, CSV upload, and reproducible Python export):
Compare conversion rates between two variants.
Pre-loaded Business Scenarios: StudyFlow Newsletter CTA Test · Retargeting Banner Offer Test · Loyalty Partner Offer Test · Social Post CTA Test (Raw Data)
Plan n for comparing two groups.
Plan n for A/B/C/D tests.
Compare before/after scores for the same people (e.g., Pre vs. Post).
Pre-loaded Business Scenarios: Global Streaming Launch Creative Test · Retail Promotions Pricing Test · B2B Nurture Content Tracks
Compare averages across 3+ groups (e.g., 4 different landing pages).
Pre-loaded Business Scenarios: Streaming Launch: Creative Concepts and Revenue Per Thousand Households · Flagship Retail Promotions: Pricing Stories and Gross Margin · B2B Nurture Tracks: Product Tutorials vs. Executive ROI Stories · Lifecycle Messaging Experiment (Raw Observations)
Analyze "switchers" in paired yes/no data (e.g., before/after purchase).
Pre-loaded Business Scenarios: Streaming Creative Test: Emotional Trailer Swap · Flagship Retail Signage: Price Messaging at Store Level · B2B Nurture Tracks: Paired Tutorial vs. Executive Story · Lifecycle Push Pilot: Paired Observations (Raw CSV Mode)
Estimate whether a campaign, launch or price change really moved results, by comparing the markets that got it with similar markets that didn't.
Pre-loaded Business Scenarios: The Textbook App Launch: In-Store Mobile App Rollout · Hearthstone Coffee: Regional TV Campaign Flight · Pinnacle Fitness: Member App Rollout & The Selection Trap · Oakline Furniture: Showroom Remodels & The Transition Dip
Check if two categorical variables are related (e.g., Device vs. Signup).
Pre-loaded Business Scenarios: Holiday Email Creative Test · Channel vs. Purchase Behavior Test · Landing Page Layout Test · Loyalty Nudge Test (Raw Data) · D&D Class Survival Test
Race a classic A/B test against adaptive bandit algorithms on the same visitors. See what each one earns during the test and what each one can tell you after it.