Marketing Analytics Statistical Glossary (469 Metrics Explained for Business Decisions)

🎯 The Manager's Question: What share of the predicted categories were right?

📊 Worked Business Example: At the 0.50 cutoff the influencer model gets 730 of 1,000 posts right, 73.0%. Always guessing "no sale" would already get 537 right, so the model adds 19.3 points.

⚖️ How to Judge the Number: Always read it next to the baseline of always guessing the most common outcome, 53.7% here. 90% sounds strong until you learn that the baseline is 92%. It moves with the cutoff, so compare models at the same cutoff and on holdout rows rather than the fitting rows.

⚠️ Common Misread to Avoid: Reading high accuracy as a model that finds the customers you care about. With a rare outcome, a model that never predicts it can score 92% while catching nobody. Check sensitivity and precision before trusting it.

📐 Technical Definition: With several categories it is correct predictions divided by rows with a known outcome. For a yes-or-no model it uses the cutoff in use, 0.50 unless you moved the slider.

🎯 The Manager's Question: What shape can each unit bend into, and what happens past the data?

📊 Worked Business Example: Wrenfield’s markets never spent more than $25,000 on social. Set every held-back market to $60,000 and the partial dependence chart averages $220,000 of quarterly revenue for the tanh network and $254,000 for the ReLU network, trained on the same data.

⚖️ How to Judge the Number: tanh is an S-curve between −1 and 1 that levels off; ReLU is 0 below 0 and a straight line above it. Inside the data the two score about the same: Wrenfield’s holdout RMSE is $12,400 for tanh and $12,700 for ReLU. Read the choice against the partial dependence chart’s shaded stretch, where they part ways.

⚠️ Common Misread to Avoid: The choice is not cosmetic. Past the training range it decides whether predictions flatten (tanh) or keep climbing in a straight line (ReLU), and neither knows which the market does.

📐 Technical Definition: ReLU is \(\max(0, z)\). Starting weights are Glorot uniform for tanh and He uniform for ReLU, with biases at 0, so each choice starts on a scale that suits it.

🎯 The Manager's Question: What share of team accounts are still paying?

📊 Worked Business Example: CollabSpace starts with 90.0% of teams active. After an 18% increase announced at Year 5, 82.0% are active right away and 80.8% when the run settles at Year 14, against 89.9% with no increase.

⚖️ How to Judge the Number: The default starts at 90%. The share drops by about 0.45 × h at the announcement, and any further fall is the network effect: teams leaving because their neighbors left. Read it next to the Revenue Index, since a smaller base at a higher price can still bring in more money, up to a point.

⚠️ Common Misread to Avoid: Assuming the cancellations on announcement day are the final count. At the default 18% increase, about 8% cancel at once and another 1.3 points follow over nine years. At 21% the follow-on cancellations never stop, and every team eventually leaves.

📐 Technical Definition: After the increase a team keeps paying only while at least \(\tau + h/2\) of its neighbors are active, and below that it leaves with a yearly chance of \(1 - \mu\). Cancelled teams never return in this case.

🎯 The Manager's Question: Will our search ad show at all, and how high on the results page?

📊 Worked Business Example: Brand A bids $2.00 on "dry skin cream" and Brand B bids $1.50. If B's ad and landing page score much better on auction-time quality, B can still take the top slot.

⚖️ How to Judge the Number: Google does not report it as a number, so judge it by what it produces. An ad below the platform's threshold does not show at all, whatever the bid. Read impression share next to cost per click: if a 10% higher bid adds few top-slot impressions, work on ad quality instead.

⚠️ Common Misread to Avoid: Reading it as bid × the displayed Quality Score, a rule for every ad auction. Google combines the bid with auction-time quality, thresholds, context and expected asset impact. Display exchanges use their own rules; in the Programmatic Ad Placement Simulator the highest eligible bid wins.

📐 Technical Definition: Platform-specific: Google computes it at each auction and publishes the inputs, not the formula. The programmatic simulator does not use it; its teaching auction compares submitted CPM bids against a floor.

🎯 The Manager's Question: How many different people saw the campaign at least once?

📊 Worked Business Example: The same 2,000,000 impressions reach 1,000,000 people at an average frequency of 2, but only 400,000 at a frequency of 5.

⚖️ How to Judge the Number: It runs from 0 up to the size of the target audience and can never exceed the impression count. Read it next to average frequency: 1,000,000 people at 2 and 400,000 people at 5 cost the same here. Which is better depends on whether the goal is new people or repetition.

⚠️ Common Misread to Avoid: Reading more impressions as more people reached. A fixed impression total spent on repetition reaches fewer people: going from frequency 2 to 5 cuts reach from 1,000,000 to 400,000.

📐 Technical Definition: Platforms estimate it from identifiers, so a person on several devices can be counted more than once. The simulator's figure is an estimate from its two inputs, not a measured count.

🎯 The Manager's Question: After a penalty for each predictor, how much of the variation in {outcome} does the model account for?

📊 Worked Business Example: The ad-campaign line has an R² of 0.867 and an adjusted R² of 0.865: with one predictor and 60 rows, the penalty is tiny. Ten useless predictors on the same 60 rows could lift R² while adjusted R² fell.

⚖️ How to Judge the Number: It is at most R², and 0 means no better than predicting the average. Use it to compare models with different numbers of predictors on the same rows: the higher one earns its extra terms. For a single model, read plain R² next to it.

⚠️ Common Misread to Avoid: Reading it as the share of variation explained. It is R² with a penalty for each extra coefficient, so it can fall when a predictor adds little. It can even go below 0.

📐 Technical Definition: It matches statsmodels rsquared_adj and the "Adjusted R-squared" line in R's summary. A predictor raises it only when its t statistic is above 1 in absolute value.

🎯 The Manager's Question: After a penalty for each predictor, how much of the variation in {outcome} does the model account for?

📊 Worked Business Example: The ad-campaign line has an R² of 0.867 and an adjusted R² of 0.865: with one predictor and 60 rows, the penalty is tiny. Ten useless predictors on the same 60 rows could lift R² while adjusted R² fell.

⚖️ How to Judge the Number: It is at most R², and 0 means no better than predicting the average. Use it to compare models with different numbers of predictors on the same rows: the higher one earns its extra terms. For a single model, read plain R² next to it.

⚠️ Common Misread to Avoid: Reading it as the share of variation explained. It is R² with a penalty for each extra coefficient, so it can fall when a predictor adds little. It can even go below 0.

📐 Technical Definition: It matches statsmodels rsquared_adj and the "Adjusted R-squared" line in R's summary. A predictor raises it only when its t statistic is above 1 in absolute value.

🎯 The Manager's Question: How long does one week of advertising keep lifting sales after it runs?

📊 Worked Business Example: In the default Brightwell model TV keeps 60% of each week's effect into the next week, a half-life of 1.4 weeks. Search keeps 10%, a half-life of 0.3 weeks, so its effect is nearly gone before the next week starts.

⚖️ How to Judge the Number: 0 weeks means the effect ends in the week the money is spent. Meta's open-source MMM, Robyn, suggests searching weekly retention of 0.3 to 0.8 for TV and 0 to 0.3 for digital channels. For TV that means half-lives of about 0.6 to 3 weeks. Read it next to the channel's contribution, and judge TV over a 3 to 4 week window, not the week it airs.

⚠️ Common Misread to Avoid: Reading a 1.4-week half-life as "the ad stops working after 1.4 weeks". It is the point where half the effect is left; a quarter is still left at 2.7 weeks and an eighth at 4.1. Added up over all weeks, TV's carryover makes a week of spend worth 2.5 times its first-week effect.

📐 Technical Definition: The tool uses geometric adstock, \(A_t = x_t + \lambda A_{t-1}\), over the full series before the holdout is cut, so carryover flows into the holdout weeks. A week of spend keeps \(\lambda^w\) of its effect after w weeks, and the half-life solves \(\lambda^t = 0.5\). Guided mode sets the retention with a slider; auto-fit searches 0 to 0.9 in steps of 0.1.

🎯 The Manager's Question: Is another lag worth its extra complexity, or does the simpler model do as well?

📊 Worked Business Example: In Kestrel's lag table one lag drops BIC from 620.3 to 519.9, and a second lag raises it again to 524.5. BIC therefore picks one day of memory.

⚖️ How to Judge the Number: Lower is better, and only differences matter: a gap under about 2 is a tie, and one over 10 is decisive. Compare only models fitted to exactly the same periods. BIC charges more per term than AIC once there are more than 7 periods, so it often picks the smaller model. Read it with the lag table's LR p-value, which asks the same question as a test.

⚠️ Common Misread to Avoid: Reading an AIC of 502 as good or bad on its own. The number grows with the number of periods, so it means nothing without a rival model. Comparing fits that used different periods is just as empty, which is why the lag table fits every row from the fifth period on.

📐 Technical Definition: \(\text{BIC} = k\ln n - 2\ln L\), where \(n\) is the number of periods fitted. The lag table fits every row on the periods from the fifth onward, so all rows share the same \(n\). The tool's reference tests check both values against statsmodels' aic and bic.

🎯 The Manager's Question: Is another lag worth its extra complexity, or does the simpler model do as well?

📊 Worked Business Example: In Kestrel's lag table one lag drops BIC from 620.3 to 519.9, and a second lag raises it again to 524.5. BIC therefore picks one day of memory.

⚖️ How to Judge the Number: Lower is better, and only differences matter: a gap under about 2 is a tie, and one over 10 is decisive. Compare only models fitted to exactly the same periods. BIC charges more per term than AIC once there are more than 7 periods, so it often picks the smaller model. Read it with the lag table's LR p-value, which asks the same question as a test.

⚠️ Common Misread to Avoid: Reading an AIC of 502 as good or bad on its own. The number grows with the number of periods, so it means nothing without a rival model. Comparing fits that used different periods is just as empty, which is why the lag table fits every row from the fifth period on.

📐 Technical Definition: \(\text{BIC} = k\ln n - 2\ln L\), where \(n\) is the number of periods fitted. The lag table fits every row on the periods from the fifth onward, so all rows share the same \(n\). The tool's reference tests check both values against statsmodels' aic and bic.

🎯 The Manager's Question: How much risk of a false alarm am I willing to accept?

📊 Worked Business Example: Aura sets alpha at 0.05 before the test, so the influencer video wins only if p falls below 0.05. Test 20 creatives that are no better than the baseline and about 1 will clear that line by chance.

⚖️ How to Judge the Number: Set to {alpha} here; 0.05 is the convention and 0.01 is stricter. Lower makes false alarms rarer and real effects harder to catch, so weigh it against power, which planners usually set at 80%. Choose it before looking, from the cost of a false win against the cost of a missed one.

⚠️ Common Misread to Avoid: Reading alpha as the chance that this particular conclusion is wrong. It is the false-alarm rate of the rule, across all the tests in which nothing real is going on. Whether this conclusion is wrong also depends on how many of the ideas you test are real.

📐 Technical Definition: Testing \(m\) hypotheses at \(\alpha\) each pushes the chance of at least one false alarm to \(1-(1-\alpha)^m\), about 0.40 for 10 tests at 0.05. The Holm and Benjamini-Hochberg corrections address this, and the bandit race's A/B planner splits alpha across its comparisons.

🎯 The Manager's Question: Would the scale be more reliable without this one item?

📊 Worked Business Example: In the default survey the overall alpha is 0.69. Every item's alpha if deleted sits between 0.63 and 0.67 except the sports item, at 0.76, so that is the one item worth dropping.

⚖️ How to Judge the Number: Compare it with the overall alpha. Below it, the item is helping; above it, the scale is more consistent without the item. The tool marks any rise above 0.005, though a rise of a few hundredths is what matters. Read it next to the corrected item-total correlation and the item's wording before dropping anything.

⚠️ Common Misread to Avoid: Dropping every item whose removal nudges alpha up, one after another. Each cut can lift alpha a little while the scale loses parts of the idea it was meant to cover. Drop one item at a time, rerun, and keep any item whose wording clearly belongs.

📐 Technical Definition: The tool recomputes alpha exactly on the remaining items, with the same complete responses. It is the quantity SPSS prints as "Cronbach's Alpha if Item Deleted", and it matches the Python reference values to six decimals.

🎯 The Manager's Question: Which respondents could reasonably belong to more than one segment?

📊 Worked Business Example: A GreenLeaf respondent at 0.52 for one segment and 0.46 for another is flagged ambiguous, because the top probability is below 0.70. If 8% of the study was ambiguous and 20% of a new file is, the new respondents fit the segments less well.

⚖️ How to Judge the Number: The flag goes up when the top probability is below 70%, a common rule of thumb for a clear latent class assignment. Read the ambiguous share in a new file against the same share in the study: a rise means more new respondents fall between segments, or more left answers blank. Send ambiguous respondents the general message rather than a segment-specific one.

⚠️ Common Misread to Avoid: Treating "ambiguous" as an error or a bad respondent. Their answers fit two segments about equally, or they left many blank. And "not ambiguous" is not a good fit either: someone unlike every segment can still be assigned at 0.99 to the nearest one.

📐 Technical Definition: Blank answers are left out of the calculation, which pulls the probabilities toward the segment sizes and so toward ambiguity. The 0.70 line follows Nagin's rule of thumb for average posterior probabilities in latent class models.

🎯 The Manager's Question: At the end of the race, what can this approach say about how well this option converts?

📊 Worked Business Example: After the Nordvik race, the A/B test's interval for the old hero runs from 2.3% to 3.3% on 3,782 visitors. Thompson sampling gave that hero only 1,728 visitors and reports 2.3% to 3.9%, about 50% wider.

⚖️ How to Judge the Number: Judge the width, not the midpoint. Read it against the gap you need to measure: a 1.6-point band on a 3% option cannot tell 2.3% from 3.9%. A narrow interval means the option was measured; a wide one means it was barely looked at.

⚠️ Common Misread to Avoid: Reading a wide interval as a sign the option is bad. A bandit that abandoned an option early leaves it with little data, so the band is wide because nobody looked. Reporting its midpoint as a finding overstates what the race established.

📐 Technical Definition: The A/B test reports the observed rate with a 95% Wald interval. A bandit reports the middle 95% of its Beta posterior, starting from a flat Beta(1, 1) prior, which is why a barely sampled option shows a visibly wider band.

🎯 The Manager's Question: Which segment should this respondent be treated as when we send the campaign?

📊 Worked Business Example: A GreenLeaf respondent at 0.81, 0.15 and 0.04 is assigned to the first segment, with max_probability 0.81. One at 0.52 and 0.46 is assigned too, but that call is barely better than a coin flip.

⚖️ How to Judge the Number: It is the segment with the highest probability, and max_probability beside it shows how clear the call was. With 3 segments the lowest possible max_probability is 0.33. The tool flags anything below 0.70 as ambiguous; read the assignment together with that flag before targeting on it.

⚠️ Common Misread to Avoid: Treating everyone assigned to a segment as a full member. Assignment throws away the uncertainty, so a respondent at 0.52 is a best guess, not a member. Segment counts built from assignments drift from the fitted segment sizes for the same reason.

📐 Technical Definition: The tool reports the winning probability as max_probability. Scoring a new file adds assigned_segment, one probability column per segment, max_probability and the ambiguous flag. A respondent with a category the study never saw is not scored, because no segment has a probability for it.

🎯 The Manager's Question: How many compliments does one complaint outweigh when a customer listens to neighbors?

📊 Worked Business Example: At the StreamFlow defaults (τ = 30%, lock-in 20%), a weight of 1.0× lets the active share climb from 88.0% to 99.2%. At 1.25× it ends at 89.3%, and at 1.5× it collapses to 17.2%.

⚖️ How to Judge the Number: 1.0× means a complaint and a compliment count the same; 2.0× means one complaint drowns out two compliments. The response is not smooth: at the defaults the network holds at 1.25× and tips before 1.5×. Read it next to Complaint Tolerance (τ), because a higher τ moves the tipping weight up: at τ = 40% the base holds until 2.0×. Use it to judge whether reliability or delight is the safer place for the budget.

⚠️ Common Misread to Avoid: Assuming a large delighted group cancels out a small unhappy pocket one for one. StreamFlow starts with 18% delighted and 12% dissatisfied, yet at 1.5× the active share falls to 17.2%. When complaints carry extra weight, a minority of voices can set the tone.

📐 Technical Definition: A loyal customer sours when \(n_i\) exceeds τ and a delighted one when it exceeds τ plus 15 points, each with a yearly chance of \(0.5(1 - \mu)\). Churned neighbors have left the conversation and are not counted. The weight is a setting to test, not an estimate from data.

🎯 The Manager's Question: How many customers is this part of the curve actually based on?

📊 Worked Business Example: Parcel & Pantry starts with 1,587 customers at risk, has 271 at month 18 and only 41 at month 24. A point resting on 41 people drops about 3 points if five of them leave.

⚖️ How to Judge the Number: The tool fades the curve where fewer than 20 customers remain, because there a single departure moves it by at least 5% of its height. Read it next to the confidence band, which widens as this count falls. At a few hundred, like the 271 at month 18 here, each point is solid.

⚠️ Common Misread to Avoid: Reading a falling count as customers churning. Customers also leave the count by being censored: at month 24 here all 41 who remain are still subscribed when the data ends. Only churn events move the curve down.

📐 Technical Definition: It is counted before that age's churn events and censorings are removed, so it is the denominator of the hazard at that age. It falls through churn and censoring alike.

🎯 The Manager's Question: What share of eligible completers followed an explicit reading instruction embedded inside the survey?

📊 Worked Business Example: In Bramblecup Cold Brew, row Q4_6 instructs respondents to select Strongly Disagree (1). Exactly 296 of 340 eligible completers selected 1, giving an 87.1% pass rate.

⚖️ How to Judge the Number: 100% means every eligible completer passed the check. Pass rates below 85% indicate substantial inattentive clicking, read alongside completion speed and straightlining.

⚠️ Common Misread to Avoid: Leaving an embedded matrix trap row inside the construct scale when calculating Cronbach's alpha. Always classify the trap row as an IMC so its instructed answer does not corrupt scale reliability.

📐 Technical Definition: Instructional manipulation checks test whether a respondent reads question stems rather than clicking heuristically. Failing a clear directed-response item provides direct evidence of inattentive responding.

🎯 The Manager's Question: Does the score rank the cases that turned out "yes" above the ones that turned out "no"?

📊 Worked Business Example: On Kestrel's 56 held-back days the model scores 0.81: pick one stockout day and one stocked day at random, and 81 times in 100 the stockout day had the higher forecast. Momentum only scores 0.62, a weak ranking.

⚖️ How to Judge the Number: 0.5 is a coin toss and 1 is a perfect ranking; 0.7 to 0.8 is useful and above 0.8 is strong for marketing data. Read it next to calibration, because AUC checks only the order of the forecasts, not their size. With few "yes" cases held back (Kestrel has 11), one case moves it a lot.

⚠️ Common Misread to Avoid: Reading a high AUC as a sign the probabilities are right. A model that forecasts every day at twice its true chance ranks the days in the same order, so its AUC is unchanged. Check the calibration chart before multiplying a forecast by dollars.

📐 Technical Definition: Equivalently, it is the probability that a randomly chosen "yes" case got a higher forecast than a randomly chosen "no" case, with ties counted as half. It does not depend on any threshold. Dynamic Logistic Regression's value is checked against scikit-learn's roc_auc_score in its reference tests.

🎯 The Manager's Question: When the model misses high one period, does it tend to miss high the next period too?

📊 Worked Business Example: In the weekly ad-sales preset the lag-1 autocorrelation of the residuals is 0.67 across 104 weeks. The chance band is only about plus or minus 0.2, so the carry-over is real.

⚖️ How to Judge the Number: 0 means no carry-over. A value further from 0 than about 2 divided by the square root of the number of rows is more than chance. Read it next to Durbin-Watson, which is about 2(1 - r1), and only when rows are in time order.

⚠️ Common Misread to Avoid: Thinking carry-over biases the slope. The slope stays centred, but the standard errors come out too small, so effects look more certain than they are. At 0.67 the true uncertainty is much wider than the coefficient table shows.

📐 Technical Definition: The rough chance band is \(\pm 2/\sqrt{n}\), where n is the number of rows. The tool sorts the residuals by the time column before computing it. Ljung-Box extends the same idea to several lags at once.

🎯 The Manager's Question: Does each factor hold its own items tightly and stay distinct from the other factors?

📊 Worked Business Example: In NovaBrew, once the three low-communality items are dropped, Social Image has AVE 0.65 and CR 0.85. Its square root of AVE, 0.80, is well above its largest correlation with another factor, 0.35, so it passes Fornell-Larcker.

⚖️ How to Judge the Number: An AVE of 0.50 is the line: below it, a factor explains less of its items than error does. A CR of 0.70 is the reliability line, read like alpha. For discriminant validity, read the square root of AVE against the factor's correlations with the other factors; it should beat every one. A failure means two factors overlap too much to report as separate ideas.

⚠️ Common Misread to Avoid: Treating CR as a second copy of alpha. CR is built from the factor loadings, so strong items count for more, and here it is the same number as that factor's omega. When CR and alpha differ, the factor's items have unequal loadings.

📐 Technical Definition: \(\text{CR} = (\sum \lambda_i)^2 / [(\sum \lambda_i)^2 + \sum (1 - \lambda_i^2)]\) over the same items. Fornell-Larcker passes when \(\sqrt{\text{AVE}}\) exceeds every absolute factor correlation \(\lvert\phi_{jk}\rvert\). The loadings come from the exploratory solution, so these are screens, and the tool does not compute HTMT, the stricter ratio the chapter recommends. Fornell-Larcker is a comparison rule, not a statistic of its own; it is listed here because the tool's Fornell-Larcker button opens this entry.

🎯 The Manager's Question: On average, how many times did each person we reached see the ad?

📊 Worked Business Example: The $10,000 plan serves 2,000,000 impressions. At an average of 2 per person it reaches 1,000,000 people; at 5 per person, only 400,000.

⚖️ How to Judge the Number: The floor is 1, since everyone reached saw the ad at least once. The textbook's rule of thumb for paid social is 2 to 3, with above 5 a sign of fatigue. Read it next to reach: for a fixed impression total, each step up in frequency means fewer different people.

⚠️ Common Misread to Avoid: Reading an average of 2 as "everyone saw it twice". Some people may have seen it 8 times and many only once, and an average is not a cap.

📐 Technical Definition: Both counts must use the same period and identity rule. The simulator holds impressions at 2,000,000 and takes frequency as the input, so reach = impressions ÷ frequency.

🎯 The Manager's Question: Over the whole period, how many sales dollars came back for each dollar spent on this channel?

📊 Worked Business Example: In the default Brightwell model, search returned $4.88 in sales per dollar and TV $2.43. By this measure search looks like the best channel, which is exactly why the CFO wants to move TV money into it.

⚖️ How to Judge the Number: 1.00 is the sales break-even: each dollar came back as one dollar of sales, which still loses money once product costs are paid. Read it next to marginal ROI. When average ROI is high and marginal ROI far lower (search: $4.88 against $1.24), the channel has already used up its easy gains.

⚠️ Common Misread to Avoid: Using average ROI to decide where the next dollar goes. The average includes a channel's first, cheapest dollars, while the next dollar lands further along the flattening curve. Search's average of $4.88 hides a next-dollar return of $1.24.

📐 Technical Definition: Both totals cover all weeks, holdout included. The return is in the outcome's units, sales dollars, not profit. A channel pays for itself only above 1 divided by the gross margin: above 2.5 at a 40% margin.

🎯 The Manager's Question: Does each factor hold its own items tightly and stay distinct from the other factors?

📊 Worked Business Example: In NovaBrew, once the three low-communality items are dropped, Social Image has AVE 0.65 and CR 0.85. Its square root of AVE, 0.80, is well above its largest correlation with another factor, 0.35, so it passes Fornell-Larcker.

⚖️ How to Judge the Number: An AVE of 0.50 is the line: below it, a factor explains less of its items than error does. A CR of 0.70 is the reliability line, read like alpha. For discriminant validity, read the square root of AVE against the factor's correlations with the other factors; it should beat every one. A failure means two factors overlap too much to report as separate ideas.

⚠️ Common Misread to Avoid: Treating CR as a second copy of alpha. CR is built from the factor loadings, so strong items count for more, and here it is the same number as that factor's omega. When CR and alpha differ, the factor's items have unequal loadings.

📐 Technical Definition: \(\text{CR} = (\sum \lambda_i)^2 / [(\sum \lambda_i)^2 + \sum (1 - \lambda_i^2)]\) over the same items. Fornell-Larcker passes when \(\sqrt{\text{AVE}}\) exceeds every absolute factor correlation \(\lvert\phi_{jk}\rvert\). The loadings come from the exploratory solution, so these are screens, and the tool does not compute HTMT, the stricter ratio the chapter recommends. Fornell-Larcker is a comparison rule, not a statistic of its own; it is listed here because the tool's Fornell-Larcker button opens this entry.

🎯 The Manager's Question: Are these items correlated at all, or could every correlation be sampling noise?

📊 Worked Business Example: For the 8 default items, Bartlett's chi-square is 144.3 on 28 degrees of freedom, with p below 0.001. That only says the items are not all unrelated: the sports item is still unrelated to every other item.

⚖️ How to Judge the Number: The test asks whether every correlation between items is 0. A p-value below 0.05 rules that out and is the minimum before factoring; above 0.05 there is nothing to factor. With 100 or more respondents it is almost always significant, so read it next to KMO, which says whether the shared correlation is enough.

⚠️ Common Misread to Avoid: Taking a significant Bartlett test as a green light for a good factor analysis. With 120 respondents, correlations of 0.25 pass it easily. It is a floor, not a quality check: KMO and the loadings do that job.

📐 Technical Definition: The degrees of freedom are \(p(p - 1)/2\), one per pair of items, and the null hypothesis is that the correlation matrix is the identity. The test assumes roughly normal items and grows with sample size. The tool matches the Python reference values to four decimals.

🎯 The Manager's Question: Are these items correlated at all, or could every correlation be sampling noise?

📊 Worked Business Example: For the 8 default items, Bartlett's chi-square is 144.3 on 28 degrees of freedom, with p below 0.001. That only says the items are not all unrelated: the sports item is still unrelated to every other item.

⚖️ How to Judge the Number: The test asks whether every correlation between items is 0. A p-value below 0.05 rules that out and is the minimum before factoring; above 0.05 there is nothing to factor. With 100 or more respondents it is almost always significant, so read it next to KMO, which says whether the shared correlation is enough.

⚠️ Common Misread to Avoid: Taking a significant Bartlett test as a green light for a good factor analysis. With 120 respondents, correlations of 0.25 pass it easily. It is a floor, not a quality check: KMO and the loadings do that job.

📐 Technical Definition: The degrees of freedom are \(p(p - 1)/2\), one per pair of items, and the null hypothesis is that the correlation matrix is the identity. The test assumes roughly normal items and grows with sample size. The tool matches the Python reference values to four decimals.

🎯 The Manager's Question: How common is the outcome among these customers?

📊 Worked Business Example: In the influencer preset 463 of 1,000 posts converted, a base rate of 46.3%. In the FitLife gym file 884 of 3,500 members churned, 25%, because churners were oversampled.

⚖️ How to Judge the Number: It is the no-model forecast: give every customer 46.3% and you match the overall rate exactly. It also sets the accuracy baseline, the larger of the rate and 1 minus it, 53.7% here. Read it next to precision, which equals the base rate when customers are flagged at random.

⚠️ Common Misread to Avoid: Reading the base rate as the accuracy to beat. A 20% rate means always guessing "no" is 80% accurate, so the bar is 80%, not 20%. It is also not a prediction for any one customer; it describes the rows being judged.

📐 Technical Definition: Rows where the outcome happened divided by rows with a known outcome. A file that oversamples the event, as the gym preset does, has a higher base rate than the real market, so its probabilities run high.

🎯 The Manager's Question: Why does a 90%-accurate churn model send alerts that are wrong most of the time?

📊 Worked Business Example: Score 1,000 subscribers at a 2% churn rate with a model that is 90% sensitive and 90% specific. It catches 18 of the 20 churners but also flags 98 of the 980 loyal customers, so only 18 of its 116 alerts are real.

⚖️ How to Judge the Number: The trap closes when the event is rare: the 10% false-alarm rate applies to 980 people, the 90% catch rate to only 20. At a 50% base rate the same model is right on 90% of its alerts; at 2%, on 15.5%. Read the precision next to the base rate, and pre-screen to a higher-risk group before automating an offer.

⚠️ Common Misread to Avoid: Blaming the false alarms on a buggy model. Nothing is broken: 10% of a large group of loyal customers outnumbers 90% of a small group of churners. The fix is to raise the base rate of the group you score, not only to tune the model.

📐 Technical Definition: Precision falls below 50% whenever \((1 - \text{Spec})(1 - \pi) > \text{Sens} \cdot \pi\), which for a rare event happens even with high sensitivity and specificity. Kahneman and Tversky (1973) showed how people neglect the base rate. Gigerenzer and Hoffrage (1995) showed that counts of people, like the lab's grid of 1,000 dots, make it easier to avoid.

🎯 The Manager's Question: What log-loss do we score knowing nothing about the customers, giving everyone the overall rate?

📊 Worked Business Example: On Email Conversion 31 of 50 subscribers converted, so giving all 50 a 62% chance scores 0.664. The best curve on engagement score gets down to 0.123, a possible improvement of 0.541.

⚖️ How to Judge the Number: It is 0.693 when the outcome splits 50/50 and falls as the outcome gets rarer: 0.325 at Luxury Test Drive's 10%. It is the bar any curve has to beat, and a curve above it is worse than knowing nothing. Read it next to the best curve's log-loss: the distance between them is what the predictor can buy. The page grades your curve on the share of that distance it captures.

⚠️ Common Misread to Avoid: Reading a low base-rate log-loss as a good model. On Luxury Test Drive the flat 10% guess scores 0.325, far below Ad Click's best curve at 0.492, yet it ranks no visitor above any other. A low number here only means the outcome is rare, so judge a curve by how far it moves below this bar.

📐 Technical Definition: It is the log-loss of the null model, whose intercept is \(\ln[\bar{y}/(1-\bar{y})]\) with a slope of 0. No other flat curve scores lower, so setting B₁ to 0 and tuning B₀ alone can only reach this bar. The lab's values match scikit-learn's log_loss on constant predictions in its reference tests.

🎯 The Manager's Question: What share of sales would the brand have made with no paid media at all?

📊 Worked Business Example: In the default Brightwell Pet Food model, base share is 64.2%: about $12.7 million of the $19.7 million sold over 104 weeks. The other 35.8% is credited to TV, search and social.

⚖️ How to Judge the Number: 0% would mean every sale came from paid media; 100% would mean media did nothing. The tool's guide expects 60% to 80% for a mature brand. Read it next to the media lift share, which is the rest, and the holdout error. A base far outside that range from a model that predicts poorly usually means a missing control, such as seasonality.

⚠️ Common Misread to Avoid: Reading a 64% base as "marketing only matters for a third of sales, so cut it". Base is what sells this year with media switched off, and much of it rests on brand memory built by past advertising. Two years of weekly data cannot see that long-run effect.

📐 Technical Definition: Base is the fitted value minus the media contributions, so each week's base plus media adds up to its fitted sales exactly. Controls such as price and promotion land in base, not media. The totals run over all weeks, holdout included.

🎯 The Manager's Question: How often would we be right with no model at all, by always guessing the most common outcome?

📊 Worked Business Example: In the influencer preset 537 of 1,000 posts made no sale, so always guessing "no sale" is right 53.7% of the time. The model's 73.0% accuracy beats it by 19.3 points.

⚖️ How to Judge the Number: It is the bar for accuracy, and for a yes-or-no outcome it is never below 50%. The gap between the model's accuracy and this figure is what the model adds to accuracy. Read it with the sensitivity, because with a rare outcome the model's real value is often in which customers it catches.

⚠️ Common Misread to Avoid: Barely beating it does not make a model useless. With a rare outcome, a model that finds half the customers who say yes can still sit within a point or two of the baseline. It is worked out on the same rows as the accuracy beside it, so it can differ between training and holdout rows.

📐 Technical Definition: The share of rows in the most common category among the rows being judged. With several categories it can sit well below 50%: three equal categories give 33%.

🎯 The Manager's Question: How far apart were test and control markets before anything launched, and does that gap matter?

📊 Worked Business Example: Before the app, test stores averaged $420K and control stores $400K, a $20K gap. DiD subtracts that gap, so the bigger stores get no credit for being bigger.

⚖️ How to Judge the Number: A gap of 0 is not required: DiD lets the groups start at different levels. What it needs is a gap that stays steady before launch, so read it next to the pre-trend check. A very large gap is a warning that the markets differ in ways that could also change their trends.

⚠️ Common Misread to Avoid: Treating a baseline gap as a broken experiment. DiD differences it out: the $20K head start never enters the $30K estimate. What breaks the design is a gap that was growing or shrinking before launch.

📐 Technical Definition: Each mean is over every market-period in its cell. The post-launch gap minus this baseline gap is the DiD estimate, which is why a steady gap cancels out.

🎯 The Manager's Question: Which category do all the other categories get compared against?

📊 Worked Business Example: The regional store preset uses Midwest as the baseline, so West shows as +13.9 and South as -9.1. Switch the baseline to South and West becomes +23.1, while every fitted value stays the same.

⚖️ How to Judge the Number: Pick the baseline that makes the gaps easiest to read, such as the largest group or the status quo. A baseline with fewer than about 5 rows makes every gap wobbly, because each one is measured against it. Read the category coefficients together with their standard errors after any switch.

⚠️ Common Misread to Avoid: Thinking a new baseline changes the model. The fit, the residuals and every diagnostic stay identical. Only the intercept and the category coefficients are re-expressed against the new baseline.

📐 Technical Definition: That simple form holds when the factor is the only predictor; with other predictors \(b_L\) is the same gap adjusted for them. Switching the baseline leaves the columns of the design matrix spanning the same space, so the fitted values and residuals are unchanged.

🎯 The Manager's Question: What share of sales would the brand have made with no paid media at all?

📊 Worked Business Example: In the default Brightwell Pet Food model, base share is 64.2%: about $12.7 million of the $19.7 million sold over 104 weeks. The other 35.8% is credited to TV, search and social.

⚖️ How to Judge the Number: 0% would mean every sale came from paid media; 100% would mean media did nothing. The tool's guide expects 60% to 80% for a mature brand. Read it next to the media lift share, which is the rest, and the holdout error. A base far outside that range from a model that predicts poorly usually means a missing control, such as seasonality.

⚠️ Common Misread to Avoid: Reading a 64% base as "marketing only matters for a third of sales, so cut it". Base is what sells this year with media switched off, and much of it rests on brand memory built by past advertising. Two years of weekly data cannot see that long-run effect.

📐 Technical Definition: Base is the fitted value minus the media contributions, so each week's base plus media adds up to its fitted sales exactly. Controls such as price and promotion land in base, not media. The totals run over all weeks, holdout included.

🎯 The Manager's Question: When a basket holds the first item, how often does it also hold the second?

📊 Worked Business Example: In Corner Market, 58.9% of baskets with pasta also hold pasta sauce, against 17.5% of all baskets. Paper towels show a higher confidence for bananas, 72.8%, only because 68.1% of all baskets hold bananas anyway.

⚖️ How to Judge the Number: Always read it against the Expected column beside it; when the two match, the first item tells you nothing. 70% confidence is impressive when the second item is in 12% of baskets and means nothing when it is in 68%. Confidence divided by expected is the rule lift, so judge the gap before choosing which item should prompt the other.

⚠️ Common Misread to Avoid: The highest-confidence rules are read as the best pairings. Any rule ending in a staple scores high, because the staple is in most baskets whatever else is there. Paper towels to bananas shows 72.8% confidence and a lift of only 1.07.

📐 Technical Definition: It is \(P(Y \mid X)\): support of both divided by support of \(X\). It is not symmetric, because the reverse rule divides by the count of \(Y\) instead. Pasta to sauce is 58.9% while sauce to pasta is 92.9%.

🎯 The Manager's Question: How many baskets actually contain everything in this rule?

📊 Worked Business Example: Pasta with pasta sauce rests on 811 of Corner Market's 5,000 baskets. Truffle oil with burrata rests on 8, so the tool hides it until the minimum is set to 8 or lower.

⚖️ How to Judge the Number: Read it first, because it is the honest measure of how much evidence there is. Below the minimum, 20 baskets by default, a pair is not mined at all; dozens is thin, hundreds is a plan. Read lift together with it: a lift of 30 on 6 baskets and a lift of 3 on 800 are different kinds of claim.

⚠️ Common Misread to Avoid: The minimum count is treated as a formality and lowered until something exciting appears. It is the one setting that decides whether the rarest rules are findings or noise: at 5, a pair on 8 baskets shows a lift of 37.6.

📐 Technical Definition: Each basket counts once, however many units it held. The minimum-baskets slider applies to every itemset the tool mines, so a three-item rule needs that many baskets holding all three.

🎯 The Manager's Question: Were these items bought together on a quick trip, or did they just land in the same big weekly shop?

📊 Worked Business Example: In Corner Market the average basket holds 7.8 items, and baskets with pasta and pasta sauce average 9.4, a ratio of 1.2. That is close to typical, so the pairing is not just a by-product of stock-up shops.

⚖️ How to Judge the Number: Read the rule's mean basket size against the store's typical basket, the second number shown. A ratio near 1 is reassuring. At 1.5, half again as many items as a typical basket, the tool marks the rule as worth questioning. Read it next to lift: a high lift that lives only in big baskets is the warning sign.

⚠️ Common Misread to Avoid: Reading a high value as "the rule is false", or a low one as "the pairing is real". It changes the reading instead: items that meet only in stock-up shops are co-present rather than paired, and a cross-sell prompt aimed at them will underperform. Lift, confidence and support cannot see this, because each counts a 3-item and a 50-item basket as one row.

📐 Technical Definition: The tool compares it with the mean over all baskets and marks a ratio of 1.5 or more. A basket of \(m\) items holds \(\binom{m}{2}\) pairs, so pair counts grow with the square of basket size while support grows only with the number of baskets. Formal treatments weight baskets by size or use a null model that holds basket sizes fixed.

🎯 The Manager's Question: What share of all baskets in the file contain this whole combination?

📊 Worked Business Example: In Corner Market, 811 of 5,000 baskets hold both pasta and pasta sauce, a support of 16.2%. Truffle oil with burrata turns up in 8 baskets, a support of 0.16%.

⚖️ How to Judge the Number: Zero means the items never share a basket. Support is a share of ALL baskets, so it is small for almost every real rule: 2% of 5,000 baskets is 100 baskets, which is plenty. Read it together with the basket count, which is what this tool filters on (20 baskets by default). It decides nothing on its own; rule lift and the q-value say whether a pair is worth a shelf.

⚠️ Common Misread to Avoid: A low support is read as a weak rule. Usually it only means neither item is a staple: truffle oil and burrata share just 0.16% of baskets, yet at nearly 38 times the chance rate. Support measures how often, not how strongly.

📐 Technical Definition: The tool counts each basket once, however many units of an item it held. It divides by every basket in the file, never only the ones holding \(X\), so single-item baskets stay in the denominator.

🎯 The Manager's Question: How much of the market already owns the product when yearly sales top out?

📊 Worked Business Example: In the colour-TV preset N* = 37,400,000 × (0.5 − 0.018 ÷ 1.34), about 18.20 million households (48.7%) at the instant sales peak. The card counts through the end of peak year 6, so it shows 23.00 million, 61.5% of m.

⚖️ How to Judge the Number: The ceiling is 50%: N* would sit at exactly half of m only if p were 0. With q/p of 37.2 it is 48.7%; at the average values p 0.03 and q 0.38 (Sultan, Farley and Lehmann, 1990) it is 46.1%. Read it next to T*, which says when that half-way point arrives.

⚠️ Common Misread to Avoid: Reading the sales peak as the moment exactly half the market has bought. Whenever p is above 0 the peak comes a little earlier: at 48.7% in the colour-TV run and 46.1% at average values.

📐 Technical Definition: It holds when q > p; otherwise sales peak at launch and N* is 0, which the tool shows. Equivalently \(F(T^*) = (q - p)/(2q)\).

🎯 The Manager's Question: Will sales dip after the early enthusiasts have bought, before mainstream buyers arrive?

📊 Worked Business Example: The Chasm lens opens on colour-TV values with no dip, so set its sliders to p 0.03, q 0.38, m 1,000,000, 25% visionaries and q21 0.002. Sales then peak at 41,000 in year 4 and sag to 37,000 by year 7, a 10.4% depth.

⚖️ How to Judge the Number: None (0%) means a single hump with no dip, which is all the plain Bass model can draw. The depth is the drop from the first peak to the trough, as a share of that peak. From the 10.4% example, raising q21 to 0.004 shrinks the dip to 3.2%, and at 0.008 it disappears. Raising the visionary share to 35% deepens it to 26.4%. Read it next to the trough year and the second peak, because a long dip is easily mistaken for a failed product.

⚠️ Common Misread to Avoid: Reading the dip as proof the product has failed. In the two-segment model it means the visionaries have run out before word of mouth reaches the mainstream; in the example, sales recover to about 73,000 a year by year 14.

📐 Technical Definition: The Chasm lens runs a two-segment model (Van den Bulte and Joshi, 2007). Visionaries respond to outside influence and to each other; the mainstream adopts from visionaries through q21 and from each other through q. The tool scans yearly sales for a peak, a lower trough and a later peak above it. The chart's Moore's chasm marker is a different idea: 16% cumulative adoption, about year 3.1 for colour TV.

🎯 The Manager's Question: Will sales dip after the early enthusiasts have bought, before mainstream buyers arrive?

📊 Worked Business Example: The Chasm lens opens on colour-TV values with no dip, so set its sliders to p 0.03, q 0.38, m 1,000,000, 25% visionaries and q21 0.002. Sales then peak at 41,000 in year 4 and sag to 37,000 by year 7, a 10.4% depth.

⚖️ How to Judge the Number: None (0%) means a single hump with no dip, which is all the plain Bass model can draw. The depth is the drop from the first peak to the trough, as a share of that peak. From the 10.4% example, raising q21 to 0.004 shrinks the dip to 3.2%, and at 0.008 it disappears. Raising the visionary share to 35% deepens it to 26.4%. Read it next to the trough year and the second peak, because a long dip is easily mistaken for a failed product.

⚠️ Common Misread to Avoid: Reading the dip as proof the product has failed. In the two-segment model it means the visionaries have run out before word of mouth reaches the mainstream; in the example, sales recover to about 73,000 a year by year 14.

📐 Technical Definition: The Chasm lens runs a two-segment model (Van den Bulte and Joshi, 2007). Visionaries respond to outside influence and to each other; the mainstream adopts from visionaries through q21 and from each other through q. The tool scans yearly sales for a peak, a lower trough and a later peak above it. The chart's Moore's chasm marker is a different idea: 16% cumulative adoption, about year 3.1 for colour TV.

🎯 The Manager's Question: How strongly do earlier buyers pull in new ones through word of mouth and visible use?

📊 Worked Business Example: In the colour-TV preset q is 0.67. Once 20% of the 37.4 million households own a set, word of mouth alone wins 0.67 × 0.20 = 13.4% of the remaining households that year.

⚖️ How to Judge the Number: At q = 0 there is no word of mouth, and sales are highest at launch and then decay. The average across published estimates is 0.38 (Sultan, Farley and Lehmann, 1990). A higher q gives a steeper takeoff and a taller peak, and an earlier peak once q is more than about 3.6 times p. Read it next to p: the ratio q/p decides whether sales build to a peak at all.

⚠️ Common Misread to Avoid: Reading q as a viral coefficient, the number of new customers each customer recruits. It is a rate: q × F(t) is the extra share of the remaining non-buyers who adopt this period. Its pull grows with the installed base but acts on a shrinking pool.

📐 Technical Definition: Because q multiplies F(t), its pull is zero at launch and strongest mid-diffusion. The tool fits p, q and m together by plain nonlinear least squares, with no pull toward published averages. A q fitted before the sales peak shifts a lot as new years of data arrive.

🎯 The Manager's Question: How fast do people adopt because of advertising and press, before friends start persuading them?

📊 Worked Business Example: In the colour-TV preset p is 0.018: in year one, outside influence alone wins about 1.8% of the 37.4 million households that will eventually buy, around 673,000 sets. On its own that pace would take decades; the q term does the rest.

⚖️ How to Judge the Number: Values run from about 0.005 to 0.05 across consumer durables, with 0.03 a common average (Sultan, Farley and Lehmann, 1990). A higher p brings the first adopters sooner and makes the sales peak earlier. Read it next to q: a product with a low p and a high q starts slowly and then takes off, which is the classic word-of-mouth shape.

⚠️ Common Misread to Avoid: Reading p as the launch-year market share. It is a rate applied to the people who have NOT yet bought. With p of 0.03 and 1,000,000 eventual buyers, year one brings about 30,000 adopters from outside influence, not 3% of the market held for good.

📐 Technical Definition: With F(0) = 0 the first period's hazard is p alone, which is why p sets the launch pace. The tool fits p, q and m to the sales history by nonlinear least squares. All three move together: a higher m pulls p down.

🎯 The Manager's Question: How many customers will eventually buy, once everyone who is going to adopt has adopted?

📊 Worked Business Example: The colour-TV preset sets m at 37,400,000 households. By year 15 cumulative sales reach 99.9% of that, and the curve flattens because almost no one is left to buy.

⚖️ How to Judge the Number: It counts eventual buyers, so check it against the addressable base: the preset's 37.4 million is a share of the households that could own a TV. Doubling m doubles every year's sales and leaves the timing unchanged. Read it next to p and q, because fitting all three to early data trades them off: a larger m comes with a smaller p.

⚠️ Common Misread to Avoid: Reading m as everyone in the target market. It counts only the people who will ever adopt: if 60% of 50 million households will eventually buy, m is 30 million, not 50 million.

📐 Technical Definition: The model treats m as fixed, so \(N(t)\) approaches m as t grows and the peak timing depends on p and q only. The Foundations lens holds m constant; real ceilings can grow with falling prices, income and population.

🎯 The Manager's Question: How many customers will eventually buy, once everyone who is going to adopt has adopted?

📊 Worked Business Example: The colour-TV preset sets m at 37,400,000 households. By year 15 cumulative sales reach 99.9% of that, and the curve flattens because almost no one is left to buy.

⚖️ How to Judge the Number: It counts eventual buyers, so check it against the addressable base: the preset's 37.4 million is a share of the households that could own a TV. Doubling m doubles every year's sales and leaves the timing unchanged. Read it next to p and q, because fitting all three to early data trades them off: a larger m comes with a smaller p.

⚠️ Common Misread to Avoid: Reading m as everyone in the target market. It counts only the people who will ever adopt: if 60% of 50 million households will eventually buy, m is 30 million, not 50 million.

📐 Technical Definition: The model treats m as fixed, so \(N(t)\) approaches m as t grows and the peak timing depends on p and q only. The Foundations lens holds m constant; real ceilings can grow with falling prices, income and population.

🎯 The Manager's Question: How fast do people adopt because of advertising and press, before friends start persuading them?

📊 Worked Business Example: In the colour-TV preset p is 0.018: in year one, outside influence alone wins about 1.8% of the 37.4 million households that will eventually buy, around 673,000 sets. On its own that pace would take decades; the q term does the rest.

⚖️ How to Judge the Number: Values run from about 0.005 to 0.05 across consumer durables, with 0.03 a common average (Sultan, Farley and Lehmann, 1990). A higher p brings the first adopters sooner and makes the sales peak earlier. Read it next to q: a product with a low p and a high q starts slowly and then takes off, which is the classic word-of-mouth shape.

⚠️ Common Misread to Avoid: Reading p as the launch-year market share. It is a rate applied to the people who have NOT yet bought. With p of 0.03 and 1,000,000 eventual buyers, year one brings about 30,000 adopters from outside influence, not 3% of the market held for good.

📐 Technical Definition: With F(0) = 0 the first period's hazard is p alone, which is why p sets the launch pace. The tool fits p, q and m to the sales history by nonlinear least squares. All three move together: a higher m pulls p down.

🎯 The Manager's Question: In which year will new-customer sales reach their high point?

📊 Worked Business Example: For the colour-TV preset T* = ln(0.67 ÷ 0.018) ÷ (0.018 + 0.67) = 3.62 ÷ 0.688, about 5.26 years. That falls inside year 6, so the Peak Year card shows Yr 6, with about 6.50 million sets.

⚖️ How to Judge the Number: With q at or below p there is no later peak: sales are highest at launch, and the scorecard shows Yr 0 (Decay). At the average values p 0.03 and q 0.38 (Sultan, Farley and Lehmann, 1990), T* is about 6.2 years. Read it next to peak sales (S*). Raising p always brings the peak sooner; raising q does too once q is more than about 3.6 times p, which covers most real products.

⚠️ Common Misread to Avoid: Reading T* as the year sales collapse. It is the turning point where yearly sales stop rising; in the colour-TV run year 7 still sells 5.50 million sets, against 6.50 million in year 6.

📐 Technical Definition: This is the continuous-time peak, which the card's note prints as the closed form; the card itself shows the tallest yearly bar. When sales dip, the Saddle Depth card lists both peaks, such as Peak Yr 4, trough Yr 7, peak Yr 14. The closed form exists only when q > p. At T* cumulative adoption is \(F(T^*) = \tfrac{1}{2} - \tfrac{p}{2q}\), just under half the market.

🎯 The Manager's Question: How many units must we be ready to supply in the single busiest year?

📊 Worked Business Example: For the colour-TV preset S* = 37,400,000 × 0.688² ÷ (4 × 0.67), about 6.61 million sets a year, as the card's note shows. The card itself shows the biggest calendar year, year 6, at 6.50 million.

⚖️ How to Judge the Number: It is a share of m: here 6.61 million is 17.7% of the 37.4 million eventual buyers. A higher q makes the peak sharper; at q = 0.22 the same market peaks at only 2.41 million a year. Read it next to T*, because capacity sized to S* sits idle once sales pass the peak.

⚠️ Common Misread to Avoid: Reading S* as the sales level to plan for from then on. After the peak, first-time sales fall: the colour-TV run drops from 6.50 million in year 6 to 1.30 million by year 10.

📐 Technical Definition: It is the instantaneous rate at T*, so it sits slightly above the best single year's total. The model counts first purchases only; replacement and repeat sales are not in it.

🎯 The Manager's Question: How strongly do earlier buyers pull in new ones through word of mouth and visible use?

📊 Worked Business Example: In the colour-TV preset q is 0.67. Once 20% of the 37.4 million households own a set, word of mouth alone wins 0.67 × 0.20 = 13.4% of the remaining households that year.

⚖️ How to Judge the Number: At q = 0 there is no word of mouth, and sales are highest at launch and then decay. The average across published estimates is 0.38 (Sultan, Farley and Lehmann, 1990). A higher q gives a steeper takeoff and a taller peak, and an earlier peak once q is more than about 3.6 times p. Read it next to p: the ratio q/p decides whether sales build to a peak at all.

⚠️ Common Misread to Avoid: Reading q as a viral coefficient, the number of new customers each customer recruits. It is a rate: q × F(t) is the extra share of the remaining non-buyers who adopt this period. Its pull grows with the installed base but acts on a shrinking pool.

📐 Technical Definition: Because q multiplies F(t), its pull is zero at launch and strongest mid-diffusion. The tool fits p, q and m together by plain nonlinear least squares, with no pull toward published averages. A q fitted before the sales peak shifts a lot as new years of data arrive.

🎯 The Manager's Question: Is growth coming mainly from advertising and launch buzz, or from customers persuading each other?

📊 Worked Business Example: The colour-TV preset has 0.67 ÷ 0.018 = 37.2, and its scorecard shows outside influence winning only 9.8% of all adopters. At the average values 0.38 and 0.03 (Sultan, Farley and Lehmann, 1990) the ratio is about 13 and that share is 20.6%.

⚖️ How to Judge the Number: A ratio of 1 is the dividing line. At or below 1 sales are highest at launch and decline; above 1 they build to a later peak. The bigger the ratio, the smaller the share of adopters won by outside influence: 9.8% at 37.2, 20.6% at about 13. Read it next to p and q themselves: the ratio sets the shape, while their sum sets how fast the curve plays out.

⚠️ Common Misread to Avoid: Reading a high ratio as proof that advertising was wasted. In the model q has nothing to multiply until p brings in the first buyers; set p to 0 and no one ever adopts.

📐 Technical Definition: Only the ratio shapes the curve. The sum p + q sets the time scale, as in \(T^* = \ln(q/p)/(p+q)\). A peak after launch needs q/p above 1. The card's lifetime outside share is \((p/q)\ln(1 + q/p)\).

🎯 The Manager's Question: Is growth coming mainly from advertising and launch buzz, or from customers persuading each other?

📊 Worked Business Example: The colour-TV preset has 0.67 ÷ 0.018 = 37.2, and its scorecard shows outside influence winning only 9.8% of all adopters. At the average values 0.38 and 0.03 (Sultan, Farley and Lehmann, 1990) the ratio is about 13 and that share is 20.6%.

⚖️ How to Judge the Number: A ratio of 1 is the dividing line. At or below 1 sales are highest at launch and decline; above 1 they build to a later peak. The bigger the ratio, the smaller the share of adopters won by outside influence: 9.8% at 37.2, 20.6% at about 13. Read it next to p and q themselves: the ratio sets the shape, while their sum sets how fast the curve plays out.

⚠️ Common Misread to Avoid: Reading a high ratio as proof that advertising was wasted. In the model q has nothing to multiply until p brings in the first buyers; set p to 0 and no one ever adopts.

📐 Technical Definition: Only the ratio shapes the curve. The sum p + q sets the time scale, as in \(T^* = \ln(q/p)/(p+q)\). A peak after launch needs q/p above 1. The card's lifetime outside share is \((p/q)\ln(1 + q/p)\).

🎯 The Manager's Question: How many units must we be ready to supply in the single busiest year?

📊 Worked Business Example: For the colour-TV preset S* = 37,400,000 × 0.688² ÷ (4 × 0.67), about 6.61 million sets a year, as the card's note shows. The card itself shows the biggest calendar year, year 6, at 6.50 million.

⚖️ How to Judge the Number: It is a share of m: here 6.61 million is 17.7% of the 37.4 million eventual buyers. A higher q makes the peak sharper; at q = 0.22 the same market peaks at only 2.41 million a year. Read it next to T*, because capacity sized to S* sits idle once sales pass the peak.

⚠️ Common Misread to Avoid: Reading S* as the sales level to plan for from then on. After the peak, first-time sales fall: the colour-TV run drops from 6.50 million in year 6 to 1.30 million by year 10.

📐 Technical Definition: It is the instantaneous rate at T*, so it sits slightly above the best single year's total. The model counts first purchases only; replacement and repeat sales are not in it.

🎯 The Manager's Question: In which year will new-customer sales reach their high point?

📊 Worked Business Example: For the colour-TV preset T* = ln(0.67 ÷ 0.018) ÷ (0.018 + 0.67) = 3.62 ÷ 0.688, about 5.26 years. That falls inside year 6, so the Peak Year card shows Yr 6, with about 6.50 million sets.

⚖️ How to Judge the Number: With q at or below p there is no later peak: sales are highest at launch, and the scorecard shows Yr 0 (Decay). At the average values p 0.03 and q 0.38 (Sultan, Farley and Lehmann, 1990), T* is about 6.2 years. Read it next to peak sales (S*). Raising p always brings the peak sooner; raising q does too once q is more than about 3.6 times p, which covers most real products.

⚠️ Common Misread to Avoid: Reading T* as the year sales collapse. It is the turning point where yearly sales stop rising; in the colour-TV run year 7 still sells 5.50 million sets, against 6.50 million in year 6.

📐 Technical Definition: This is the continuous-time peak, which the card's note prints as the closed form; the card itself shows the tallest yearly bar. When sales dip, the Saddle Depth card lists both peaks, such as Peak Yr 4, trough Yr 7, peak Yr 14. The closed form exists only when q > p. At T* cumulative adoption is \(F(T^*) = \tfrac{1}{2} - \tfrac{p}{2q}\), just under half the market.

🎯 The Manager's Question: How many customers does the network look at before each correction?

📊 Worked Business Example: Ridgeline’s 2,399 training rows in batches of 64 give 38 corrections per epoch. Batches of 512 would give 5 steadier ones, so each epoch would move the weights less.

⚖️ How to Judge the Number: 64 is the default and 32 to 128 is typical. Smaller batches mean more, noisier corrections per epoch; larger ones mean fewer, steadier ones. Read it together with the learning rate, since both set how jumpy the curves look.

⚠️ Common Misread to Avoid: It is not the sample size. Every training row is still used in every epoch; the batch size only sets how often the weights are corrected along the way.

📐 Technical Definition: The rows of each epoch are shuffled and split into groups of this size, and the last group may be smaller. The weights are updated once per group.

🎯 The Manager's Question: At which pass was the network best at predicting customers it was not trained on?

📊 Worked Business Example: At Ridgeline Roasters the best epoch is 22, where validation loss bottoms out at 0.529. Early stopping trains on to epoch 42, finds nothing better, and reports the network from epoch 22.

⚖️ How to Judge the Number: There is no right number; read where it falls against the maximum of 300. A best epoch below 5 suggests a learning rate so high that the curves jump, and the page flags it. One within the last 20 epochs of the maximum means the network may still be improving, so allow more. Read it next to the gap between the two curves at that point: at Ridgeline it is 0.008, so little memorizing yet.

⚠️ Common Misread to Avoid: The last epoch is not the finished model. After the best one, the network keeps improving on the customers it studies and gets worse on new ones. At Tallyhop, with early stopping off, the last epoch scores a holdout log loss of 2.00 against 0.67 at the best epoch, epoch 9.

📐 Technical Definition: Validation loss is measured after the epoch’s last update and without the weight-decay penalty. A later epoch counts as better only if it beats the best by more than \(10^{-6}\). The page flags a best epoch below 5 (stopped too early) and one inside the last patience window before the maximum (never settled).

🎯 The Manager's Question: Is another lag worth its extra complexity, or does the simpler model do as well?

📊 Worked Business Example: In Kestrel's lag table one lag drops BIC from 620.3 to 519.9, and a second lag raises it again to 524.5. BIC therefore picks one day of memory.

⚖️ How to Judge the Number: Lower is better, and only differences matter: a gap under about 2 is a tie, and one over 10 is decisive. Compare only models fitted to exactly the same periods. BIC charges more per term than AIC once there are more than 7 periods, so it often picks the smaller model. Read it with the lag table's LR p-value, which asks the same question as a test.

⚠️ Common Misread to Avoid: Reading an AIC of 502 as good or bad on its own. The number grows with the number of periods, so it means nothing without a rival model. Comparing fits that used different periods is just as empty, which is why the lag table fits every row from the fifth period on.

📐 Technical Definition: \(\text{BIC} = k\ln n - 2\ln L\), where \(n\) is the number of periods fitted. The lag table fits every row on the periods from the fifth onward, so all rows share the same \(n\). The tool's reference tests check both values against statsmodels' aic and bic.

🎯 The Manager's Question: Could we bid less and still win enough auctions to come out ahead?

📊 Worked Business Example: GlowRoute values the space at $5 CPM and the highest rival bids $3. A $3.20 bid still wins and pays $3.20 instead of $5, but a $2.80 bid loses and buys nothing.

⚖️ How to Judge the Number: No shading means bidding the full value, which wins most often but pays the most. The gain on a win is the value minus the bid: $5.00 − $3.20 = $1.80 per 1,000 here. Read it against the win rate over many comparable auctions, since a deeper shade saves more per win but wins less often.

⚠️ Common Misread to Avoid: Reading shading as a discount applied after the auction. The buyer lowers its bid before bidding, without seeing rival bids, and a first-price winner still pays exactly what it bid.

📐 Technical Definition: Shading software estimates the chance of winning at each bid from past auctions (Zhou et al., 2021). The simulator's bidding diagnostic replays one auction with the rival bid fixed at $3 and the rival winning ties.

🎯 The Manager's Question: If the model were fitted without this case, how surprising would its value of {outcome} be?

📊 Worked Business Example: The $6.6k campaign has an externally studentized residual of -3.34. Allowing for the 60 campaigns checked, its Bonferroni p-value is 0.088, so it is not flagged as an outlier at 0.05.

⚖️ How to Judge the Number: Read it like a t value: 0 is no surprise and beyond plus or minus 3 is unusual. The Bonferroni test multiplies the p-value by the number of rows, because checking every row gives chance many tries. An adjusted p-value below {alpha} flags the case; read it next to Cook's distance to see whether the case also moves the fit.

⚠️ Common Misread to Avoid: Treating "flagged" as "delete". Check the row for a data-entry error first. If the value is real, report the results with and without it.

📐 Technical Definition: When the model holds, \(t_i\) follows a t distribution on \(n-k-1\) degrees of freedom. The tool reports the largest one with the Bonferroni p-value \(\min(1,\ n \cdot 2P(T_{n-k-1} > |t_i|))\), the same rule as car::outlierTest in R.

🎯 The Manager's Question: How far could this number move if we had surveyed a different set of customers of the same size?

📊 Worked Business Example: With the Artisan Bean raw data loaded (350 customers), the tool refits both equations on 5,000 resamples. The indirect effect is $1.57 per price tier, and the middle 95% of the resampled values runs from $1.23 to $1.92.

⚖️ How to Judge the Number: An interval that excludes zero is the test, here at the {level}% level. Its ends are the honest margin: $1.23 to $1.92 says the quality route is worth at least about $1.20 per tier step. It need not be symmetric around the estimate, and for a product of two paths it usually is not. Read it next to the estimate and the seed, because a rerun with another seed moves each end by up to about a cent.

⚠️ Common Misread to Avoid: Resampling is read as creating new data, so 5,000 resamples look like far more evidence than 350 customers. Every resample reuses the same 350 rows. A biased or tiny sample gives a precise-looking interval around the wrong answer.

📐 Technical Definition: Assumes the rows are independent draws from the population of interest. The tool draws 1,000 to 20,000 samples (5,000 by default) of the same size with replacement, refits every equation on each and reads the percentiles. A bias-corrected and accelerated (BCa) option is offered, and the seed is fixed so a rerun matches. Refitting all equations on the same rows keeps how the paths move together, which a Monte Carlo interval has to assume away.

🎯 The Manager's Question: Are the model's misses bigger for some kinds of cases than for others?

📊 Worked Business Example: In the B2B deal-size preset the test gives LM = 6.48 on 1 degree of freedom, p = 0.011. The residual spread is 2.5 times wider for the largest third of deals than for the smallest.

⚖️ How to Judge the Number: A p-value below {alpha} says the uneven spread is more than chance. Call it practically uneven only when the widest spread is also at least twice the narrowest, a 2:1 ratio. Read it together with that spread ratio: both together are the tool's warning, and either alone is a watch.

⚠️ Common Misread to Avoid: Thinking uneven spread biases the coefficients. It breaks their standard errors, so the p-values and intervals are off. Robust standard errors or a log of the outcome repair that.

📐 Technical Definition: LM is compared with a chi-square on \(k - 1\) degrees of freedom, where k is the number of coefficients. The Koenker version does not assume normal errors, and matches statsmodels het_breuschpagan with its default robust setting.

🎯 The Manager's Question: How far, on average, were the forecast probabilities from what actually happened?

📊 Worked Business Example: Kestrel's model scores 0.093 on its 56 held-back days, against 0.129 for momentum only and 0.158 for the base rate. The model's forecasts sit closest to what happened.

⚖️ How to Judge the Number: 0 is perfect, and always forecasting 50% scores 0.25. For a rare event the base rate scores far below 0.25, so compare against it on the same periods. Read it together with log loss: the two usually agree, and when they do not, log loss is reacting to a few confident misses.

⚠️ Common Misread to Avoid: Reading 0.093 as a 9.3% error rate. It is an average of squared gaps between forecast and outcome, so it has no percentage meaning on its own. Its square root, about 0.30, is the typical size of a miss: 30 percentage points.

📐 Technical Definition: It is scored on the same forecasts as log loss, and Dynamic Logistic Regression's value is checked against scikit-learn's brier_score_loss in its reference tests. It is gentler on a confident miss: a 99% forecast that turns out "no" costs 0.98 here but 4.6 in log loss. So the two can rank close forecasts differently.

🎯 The Manager's Question: When the forecast says 30%, does it happen about 30% of the time?

📊 Worked Business Example: On Kestrel's 56 held-back days the forecasts average 18.2% and the shelf is empty on 19.6%, so they run about 1.5 points low. The 29 days forecast at 10% to 20% (10.8% on average) ran out on 6.9%.

⚖️ How to Judge the Number: Points on the diagonal mean a forecast of 30% happens 30 times in 100. Points below it mean the model overstates the chance, and points above it mean it understates. Dynamic Logistic Regression flags an average forecast more than 5 points from the actual rate, or a bin of 20 or more periods more than 15 points off the diagonal. Read it alongside AUC: ranking and calibration are separate checks, and a budget that multiplies probabilities by dollars needs this one.

⚠️ Common Misread to Avoid: Reading one bin that misses the diagonal as a broken model. Kestrel's 40% to 50% bin holds 3 days: 2 empty shelves against 1.3 forecast, which is noise. Judge the pattern across bins, and give most weight to the bins with the most periods.

📐 Technical Definition: Dynamic Logistic Regression, Panel Dynamic Logit and the neural network lab use ten equal-width bins (0 to 10%, 10% to 20%, and so on). They drop bins with fewer than three periods or rows. The oversampling lab instead uses ten groups of equal size, with no minimum. Each bin is plotted at its average forecast against its observed share of "yes". A calibrated model satisfies the display line in every bin. One summary, calibration-in-the-large, compares the average forecast with the overall rate across every held-back period or row.

🎯 The Manager's Question: How long does one week of advertising keep lifting sales after it runs?

📊 Worked Business Example: In the default Brightwell model TV keeps 60% of each week's effect into the next week, a half-life of 1.4 weeks. Search keeps 10%, a half-life of 0.3 weeks, so its effect is nearly gone before the next week starts.

⚖️ How to Judge the Number: 0 weeks means the effect ends in the week the money is spent. Meta's open-source MMM, Robyn, suggests searching weekly retention of 0.3 to 0.8 for TV and 0 to 0.3 for digital channels. For TV that means half-lives of about 0.6 to 3 weeks. Read it next to the channel's contribution, and judge TV over a 3 to 4 week window, not the week it airs.

⚠️ Common Misread to Avoid: Reading a 1.4-week half-life as "the ad stops working after 1.4 weeks". It is the point where half the effect is left; a quarter is still left at 2.7 weeks and an eighth at 4.1. Added up over all weeks, TV's carryover makes a week of spend worth 2.5 times its first-week effect.

📐 Technical Definition: The tool uses geometric adstock, \(A_t = x_t + \lambda A_{t-1}\), over the full series before the holdout is cut, so carryover flows into the holdout weeks. A week of spend keeps \(\lambda^w\) of its effect after w weeks, and the half-life solves \(\lambda^t = 0.5\). Guided mode sets the retention with a slider; auto-fit searches 0 to 0.9 in steps of 0.1.

🎯 The Manager's Question: How many agents moved or switched state in the most recent simulated year?

📊 Worked Business Example: The default Brookhaven run moves 896 households in Year 1, 41 in Year 10 and just 1 a year from Year 17. At Year 21 no household is below τ and the run settles.

⚖️ How to Judge the Number: 0 means the market is at rest, either because every agent is content or because nobody who wants to change can. A falling count signals an approaching rest; a spike after a shock shows how far the disturbance travelled. Read it next to Cascade Momentum, its year-on-year ratio, and the outcome line, which says whether the moves are changing anything.

⚠️ Common Misread to Avoid: Reading zero transitions as a healthy market. A town can freeze fully sorted, and a subscription base can freeze after half its teams have left. Zero means nothing is changing, not that the end state is good.

📐 Technical Definition: The count of agents whose state, or in the housing case whose lot, differs from the year before. Changes made by a shock itself are not included. The CX, discount and viral-ad runs settle after 5 years in a row with no transitions, or 10 years with the outcome inside a 1-point band.

🎯 The Manager's Question: How much of the model's evidence does each customer group supply?

📊 Worked Business Example: In the influencer preset 47.8% of posts are Micro tier, 36.6% Mid and 15.6% Mega. The Mega comparison therefore rests on 156 posts, against 478 for Micro.

⚖️ How to Judge the Number: Shares add to 100% within a predictor. A level with only a few dozen rows, or fewer than about 10 of either answer, gives the model little to estimate its gap from the reference, so expect a wide interval. Read it alongside that level's interval in the coefficient table.

⚠️ Common Misread to Avoid: Reading the share as the group's conversion rate. It counts membership, not outcomes: Mega is 15.6% of posts, yet 73% of Mega posts converted. Look up outcomes by level in the predicted probabilities instead.

📐 Technical Definition: 100 times the rows in the level divided by the usable rows for that predictor.

🎯 The Manager's Question: How many customers were still active when the data ended, so we cannot yet say when they leave?

📊 Worked Business Example: Parcel & Pantry follows 1,587 customers, and 455 of them (28.7%) were still subscribed in the last month of data. Each counts as a survivor for every month they were watched, then drops out of the denominator.

⚖️ How to Judge the Number: 0% means every customer has finished their story and a simple churn rate is safe. The tool warns above 60% censored: past that, report retention only up to an age your data reaches, never a lifetime. Read it next to the three churn rates under "How honest is your churn rate?", which drift apart as censoring grows.

⚠️ Common Misread to Avoid: Counting censored customers as retained, or dropping them, both bias the churn rate. At month 18 here, counting them as retained gives 69.8% churned and dropping them gives 81.3%. Kaplan-Meier keeps each one for as long as they were watched and gives 71.9%.

📐 Technical Definition: A customer censored at age \(t\) is in the at-risk count up to \(t\) and then leaves it without counting as churn. This assumes censoring is unrelated to the risk of leaving, which breaks if the customers you stop observing are the ones about to go. Under a quiet-period rule, a customer whose current quiet run is still shorter than k is censored too.

🎯 The Manager's Question: How much more money could this channel take before extra dollars stop earning anything?

📊 Worked Business Example: Email spent $2,400 this month at a true return of 20.8x. The lab caps next month's useful email spend at 3 times that, $7,200, so $30,000 put on email projects the same revenue as $7,200.

⚖️ How to Judge the Number: The cap is 3 times this month's spend for every channel, a deliberate simplification rather than a measured limit. Read it next to the true ROAS: the benchmark split fills the highest-return channel to its cap first, then the next. Dollars past the cap earn nothing in the projection.

⚠️ Common Misread to Avoid: Reading the cap as a forecast of what email could really take. It is a guard against extrapolating a 20x return across a $60,000 budget, which would project a seven-figure month from one email list. Real channels saturate, and the lab does not know where.

📐 Technical Definition: Both the student's split and the benchmark use the same capped model, so they are comparable with each other and neither is a forecast. The true ROAS is this month's revenue divided by this month's spend, from the ground-truth log rather than the dashboard.

🎯 The Manager's Question: Can the model tell these channels' effects apart, or do their budgets always move together?

📊 Worked Business Example: In the Lumen Beauty case, social and influencer spend correlate 0.96 and each has a VIF of about 9.2. The model credits social with 74% of media sales and influencer with 15%, but the data cannot really say which deserved it.

⚖️ How to Judge the Number: A VIF of 1 means a channel's spend is unrelated to the others. The tool warns when any VIF tops 5 or any pair of channels correlates above 0.80. Read it next to the channel contributions and the reallocation: under the warning, report the collinear channels' combined effect and test the split with an experiment.

⚠️ Common Misread to Avoid: Concluding that one of the collinear channels does not work. High collinearity means the model cannot tell which channel earned the credit, because they always rose and fell together. Their combined contribution can still be estimated well.

📐 Technical Definition: The VIF uses the adstocked, saturated media columns over the training weeks; the pairwise check uses raw weekly spend. A VIF of 5 means the coefficient's variance is five times what it would be if the channels were unrelated.

🎯 The Manager's Question: How many sales did each channel add on top of what would have sold anyway?

📊 Worked Business Example: In the default Brightwell model, TV contributes about $3.44 million over 104 weeks and search $3.04 million, 49% and 43% of the media total. Display contributes $0: its fitted effect came out negative, so the tool fixed it at zero.

⚖️ How to Judge the Number: 0 means the channel added nothing the model can see. Read each contribution next to what it cost: TV's $3.44 million came from $1.41 million of spend, which is its average ROI. A big contribution can still be a poor home for the next dollar, so read marginal ROI before moving money.

⚠️ Common Misread to Avoid: Treating contribution as counted sales, the way a coupon code is. It is the model's estimate of what sales would lose without the channel. When two channels' spend moves together, the model can split their credit almost arbitrarily, so check the collinearity warning first.

📐 Technical Definition: Media coefficients are fitted by least squares on the training weeks with a non-negativity rule: a channel whose coefficient comes out negative is refitted at zero and flagged. Contributions are summed over every week, holdout included. The numbers match a Python statsmodels reference on all four cases.

🎯 The Manager's Question: Do the counts across our categories differ from the benchmark split by more than chance would produce?

📊 Worked Business Example: Zenith Beverages expected 250 of its 1,250 taste testers per flavor, but Berry Blast drew 380 and Matcha Zen 130. Summed over the 5 flavors, chi-square is 154.4 on 4 degrees of freedom, far past the 9.49 cut-off.

⚖️ How to Judge the Number: Zero means the counts match the benchmark exactly. If the benchmark were true, chi-square would average its degrees of freedom, so read it against the critical value for that df. With 4 df at the 0.05 level the cut-off is 9.49. Then read the component column to see which categories drive the total, and shift production or ad spend toward the ones that over-index.

⚠️ Common Misread to Avoid: Reading a large chi-square as "every category is off". It is one number for the whole split: in Zenith's data, two of the five flavors supply 81% of it. Read the components, (O minus E) squared over E, to find which categories over- or under-perform.

📐 Technical Definition: Under the benchmark the statistic follows a chi-square distribution on \(k - 1\) degrees of freedom, provided the expected counts are large enough. Cochran's rule asks for every expected count to be at least 5, and the tool flags any below that. It matches scipy.stats.chisquare.

🎯 The Manager's Question: Do the counts across our categories differ from the benchmark split by more than chance would produce?

📊 Worked Business Example: Zenith Beverages expected 250 of its 1,250 taste testers per flavor, but Berry Blast drew 380 and Matcha Zen 130. Summed over the 5 flavors, chi-square is 154.4 on 4 degrees of freedom, far past the 9.49 cut-off.

⚖️ How to Judge the Number: Zero means the counts match the benchmark exactly. If the benchmark were true, chi-square would average its degrees of freedom, so read it against the critical value for that df. With 4 df at the 0.05 level the cut-off is 9.49. Then read the component column to see which categories drive the total, and shift production or ad spend toward the ones that over-index.

⚠️ Common Misread to Avoid: Reading a large chi-square as "every category is off". It is one number for the whole split: in Zenith's data, two of the five flavors supply 81% of it. Read the components, (O minus E) squared over E, to find which categories over- or under-perform.

📐 Technical Definition: Under the benchmark the statistic follows a chi-square distribution on \(k - 1\) degrees of freedom, provided the expected counts are large enough. Cochran's rule asks for every expected count to be at least 5, and the tool flags any below that. It matches scipy.stats.chisquare.

🎯 The Manager's Question: Do the counts across our categories differ from the benchmark split by more than chance would produce?

📊 Worked Business Example: Zenith Beverages expected 250 of its 1,250 taste testers per flavor, but Berry Blast drew 380 and Matcha Zen 130. Summed over the 5 flavors, chi-square is 154.4 on 4 degrees of freedom, far past the 9.49 cut-off.

⚖️ How to Judge the Number: Zero means the counts match the benchmark exactly. If the benchmark were true, chi-square would average its degrees of freedom, so read it against the critical value for that df. With 4 df at the 0.05 level the cut-off is 9.49. Then read the component column to see which categories drive the total, and shift production or ad spend toward the ones that over-index.

⚠️ Common Misread to Avoid: Reading a large chi-square as "every category is off". It is one number for the whole split: in Zenith's data, two of the five flavors supply 81% of it. Read the components, (O minus E) squared over E, to find which categories over- or under-perform.

📐 Technical Definition: Under the benchmark the statistic follows a chi-square distribution on \(k - 1\) degrees of freedom, provided the expected counts are large enough. Cochran's rule asks for every expected count to be at least 5, and the tool flags any below that. It matches scipy.stats.chisquare.

🎯 The Manager's Question: How often did the model's favourite option match what the respondent actually picked?

📊 Worked Business Example: With three products per task, guessing hits 33% of held-out choices. A hit rate of 60% means the model's top pick matched the real choice in 6 of every 10 tasks.

⚖️ How to Judge the Number: The anchor is chance, 1/J for tasks with J options, printed beside it. Read it next to log loss: a high hit rate with a poor log loss means the model ranks well but its probabilities are overconfident.

⚠️ Common Misread to Avoid: Reading a strong in-sample fit as evidence that the model predicts. Only tasks held out before fitting test prediction, and with five per respondent one hit more or less moves that person's rate by 20 points.

📐 Technical Definition: It is scored only on complete held-out tasks. The fractional credit for ties keeps a model that predicts equal probabilities from scoring a lucky hit.

🎯 The Manager's Question: How much probability did the model give the choices it never saw during fitting?

📊 Worked Business Example: With three products per task, guessing gives each option 1/3 and a log loss of ln 3, about 1.10. A model that puts 0.60 on each chosen option scores about 0.51.

⚖️ How to Judge the Number: Lower is better, and 0 would mean full probability on every chosen option. The anchor is the equal-choice baseline printed beside it, ln J for tasks with J options. Read it next to the hit rate: beating chance on both is the bar.

⚠️ Common Misread to Avoid: Reading log loss as an error percentage. It is the average of minus the log of the probability given to what was chosen, so 0.51 does not mean 51% wrong.

📐 Technical Definition: The held-out tasks are removed before estimation and scored with each respondent's own utilities. The equal-choice baseline averages \(\ln J\) across those same tasks. A few held-out tasks are a diagnostic, not a full validation.

🎯 The Manager's Question: What share of the customer base has cancelled and not yet been replaced?

📊 Worked Business Example: With no outage, StreamFlow ends at 0.8% churned. After an outage at Year 10, churned seats climb to 17.0% by Year 21, then ease to 9.9% by Year 60 as empty seats refill.

⚖️ How to Judge the Number: 0% means nobody has left. Each year 10% of churned seats refill with new loyal customers, so the share levels off instead of climbing for ever. Read it next to the dissatisfied share on the same card: dissatisfied customers are churn that has not happened yet.

⚠️ Common Misread to Avoid: Reading this as a churn rate for the year. It is the share of all seats that are empty right now, a stock rather than a flow. A year in which many customers cancel can still show the share falling if refills outpace them.

📐 Technical Definition: A dissatisfied customer cancels with a yearly chance of \(0.25(1 - \mu)\), 1.5 times that once a competitor is active. A churned seat refills with chance 0.10 a year. Churned neighbors drop out of the word-of-mouth count.

🎯 The Manager's Question: Which true values are still believable, given what this sample showed?

📊 Worked Business Example: Aura's 70 sales in 1,850 clicks give a 95% interval of 2.9% to 4.7%, in the card's small print under the one-sided bound. All of it sits above the 2.4% baseline, so the baseline is ruled out.

⚖️ How to Judge the Number: Check it against the no-change value: 0 for a difference, 1 for a ratio, the baseline for a rate. An interval that clears that value rules it out; one that straddles it cannot choose. Then read the width, which is the range of outcomes to cost out. Read it next to the p-value, which answers the same yes-or-no question but says nothing about size.

⚠️ Common Misread to Avoid: Reading a {level}% interval as a {level}% chance that the true value is inside this one. The {level}% describes the method: across many samples, intervals built this way catch the true value that often. This particular interval either holds it or does not, so read it as the values this data does not rule out.

📐 Technical Definition: It assumes the estimate's sampling distribution is close to the normal or t shape, which small samples and heavy skew can break. For a proportion the one-sample tool uses the Wald interval \(\hat{p} \pm z\sqrt{\hat{p}(1-\hat{p})/n}\), clipped to 0 and 1; for a mean it uses t. A one-tailed test also shows a one-sided bound: Aura's rate is at least 3.05%. Intervals for ratios, such as an odds ratio, are built on the log scale, so they sit lopsided around the estimate.

🎯 The Manager's Question: Which true values are still believable, given what this sample showed?

📊 Worked Business Example: Aura's 70 sales in 1,850 clicks give a 95% interval of 2.9% to 4.7%, in the card's small print under the one-sided bound. All of it sits above the 2.4% baseline, so the baseline is ruled out.

⚖️ How to Judge the Number: Check it against the no-change value: 0 for a difference, 1 for a ratio, the baseline for a rate. An interval that clears that value rules it out; one that straddles it cannot choose. Then read the width, which is the range of outcomes to cost out. Read it next to the p-value, which answers the same yes-or-no question but says nothing about size.

⚠️ Common Misread to Avoid: Reading a {level}% interval as a {level}% chance that the true value is inside this one. The {level}% describes the method: across many samples, intervals built this way catch the true value that often. This particular interval either holds it or does not, so read it as the values this data does not rule out.

📐 Technical Definition: It assumes the estimate's sampling distribution is close to the normal or t shape, which small samples and heavy skew can break. For a proportion the one-sample tool uses the Wald interval \(\hat{p} \pm z\sqrt{\hat{p}(1-\hat{p})/n}\), clipped to 0 and 1; for a mean it uses t. A one-tailed test also shows a one-sided bound: Aura's rate is at least 3.05%. Intervals for ratios, such as an odds ratio, are built on the log scale, so they sit lopsided around the estimate.

🎯 The Manager's Question: Which true values are still believable, given what this sample showed?

📊 Worked Business Example: Aura's 70 sales in 1,850 clicks give a 95% interval of 2.9% to 4.7%, in the card's small print under the one-sided bound. All of it sits above the 2.4% baseline, so the baseline is ruled out.

⚖️ How to Judge the Number: Check it against the no-change value: 0 for a difference, 1 for a ratio, the baseline for a rate. An interval that clears that value rules it out; one that straddles it cannot choose. Then read the width, which is the range of outcomes to cost out. Read it next to the p-value, which answers the same yes-or-no question but says nothing about size.

⚠️ Common Misread to Avoid: Reading a {level}% interval as a {level}% chance that the true value is inside this one. The {level}% describes the method: across many samples, intervals built this way catch the true value that often. This particular interval either holds it or does not, so read it as the values this data does not rule out.

📐 Technical Definition: It assumes the estimate's sampling distribution is close to the normal or t shape, which small samples and heavy skew can break. For a proportion the one-sample tool uses the Wald interval \(\hat{p} \pm z\sqrt{\hat{p}(1-\hat{p})/n}\), clipped to 0 and 1; for a mean it uses t. A one-tailed test also shows a one-sided bound: Aura's rate is at least 3.05%. Intervals for ratios, such as an odds ratio, are built on the log scale, so they sit lopsided around the estimate.

🎯 The Manager's Question: Of the customers really in this category, what share did the model put there?

📊 Worked Business Example: On the 1,050 test rows of the FitLife churn preset, the default tree puts 50 of 274 churners in Churned, a recall of 18.2%. It puts 94.8% of members who stayed in Stayed, so its 74.9% accuracy hides the gap.

⚖️ How to Judge the Number: A model that guesses at random in proportion to the shares scores each category's own share, about 26% for Churned on those test rows. Compare it across the categories: a big gap means the model favours the common outcome. Read it next to macro F1 and each category's precision.

⚠️ Common Misread to Avoid: Reading it as the share of the model's calls for this category that were right. That is precision; recall divides by the rows actually in the category. A model that puts everyone in this category scores 100% here.

📐 Technical Definition: For a yes-or-no outcome, recall for the yes category is the sensitivity and recall for the no category is the specificity.

🎯 The Manager's Question: How badly did the predicted chances miss what customers actually did, counting confident misses hardest?

📊 Worked Business Example: The influencer model's log loss is 0.532, against 0.690 for giving every post the 46.3% base rate. A single post given a 3% chance that then converted costs 3.5, as much as about seven average posts.

⚖️ How to Judge the Number: Lower is better and 0 is perfect. The bar to beat is the base rate given to everyone: about 0.69 at a 50% rate and 0.33 at a 10% rate. Read it next to the AUC, and compare training with holdout, or two models, on the same rows.

⚠️ Common Misread to Avoid: Reading 0.53 as a 53% error rate. It is an average penalty in log units, not a share of anything. It punishes confident misses far more than hesitant ones: a 10% forecast for something that happens costs 2.30, a 50% forecast only 0.69.

📐 Technical Definition: Probabilities are clipped to between \(10^{-15}\) and \(1 - 10^{-15}\) so a certain miss stays finite. With several categories it is the average of \(-\ln \hat{p}\) for the category that happened, and guessing all K equally scores \(\ln K\): 1.10 for three, 1.39 for four. The binary form equals the deviance divided by 2n.

🎯 The Manager's Question: Once our ad wins the space, how much do we actually pay for it?

📊 Worked Business Example: In the top-banner auction LuxeDerma's $4.20 CPM beats GlowRoute's $3.80 and clears the $2.50 floor. Under the first-price rule LuxeDerma pays its own $4.20 CPM, which is $0.0042 for the one impression.

⚖️ How to Judge the Number: Under the first-price rule the clearing price equals the winning bid. Read it next to the runner-up bid and the floor: LuxeDerma paid about $0.40 per 1,000 more than it needed to beat GlowRoute. A second-price rule would instead charge about the runner-up's bid, here $3.80.

⚠️ Common Misread to Avoid: Assuming the winner pays the runner-up's $3.80, as in older second-price auctions. In this first-price auction LuxeDerma pays its own $4.20, and bid shading lowers a bid before the auction, never the price after it.

📐 Technical Definition: The simulator holds fees, approval and delivery equal, so the price is the bid; real exchanges may report it gross or net of fees. A tie goes to the brand listed first. An unsold space has no clearing price and costs $0.

🎯 The Manager's Question: How big is the gap between our average and the benchmark, measured in typical spreads between cases?

📊 Worked Business Example: Sixty weekly cohorts averaged a $42.15 acquisition cost against the CFO's $45 ceiling, with a standard deviation of $7.80. Cohen's d is (42.15 minus 45) / 7.80 = -0.37, so the average sits about a third of a typical spread below the ceiling.

⚖️ How to Judge the Number: Zero means the average sits exactly on the benchmark; the sign only says which side. Read it next to the p-value and the gap in dollars: with a big enough sample, a d of 0.05 is significant but rarely worth acting on.

⚠️ Common Misread to Avoid: Treating a tiny p-value as proof of a big effect. With 20,000 rows, a d of 0.03 gives p below 0.001, yet the average has moved only 3% of one typical spread. Cohen's d does not grow with the sample; the p-value does.

📐 Technical Definition: This is the one-sample form the One-Sample Testing Lab prints. Two-group versions divide the gap between the group means by a pooled standard deviation instead. Cohen's (1988) bands are conventions for when no better yardstick exists, and a gap in dollars against a budget is better.

🎯 The Manager's Question: How large is the average within-unit change compared with the spread of individual changes?

📊 Worked Business Example: Suppose basket size rises by $2.15 on average across 20 stores and store change scores have SD = $1.27. Then d_z = 2.15 / 1.27 = 1.69.

⚖️ How to Judge the Number: Use 0.20 (small), 0.50 (medium), and 0.80 (large) alongside the raw mean change and baseline average.

⚠️ Common Misread to Avoid: Comparing d_z directly to an independent-groups Cohen’s d without checking the denominator. Because d_z divides by the SD of change scores, correlated pairs produce a larger d_z.

📐 Technical Definition: The paired t-statistic equals \(d_z \sqrt{n}\). When pre- and post-scores correlate positively, \(s_d\) is smaller than the raw group SD.

🎯 The Manager's Question: How big is the gap between our average and the benchmark, measured in typical spreads between cases?

📊 Worked Business Example: Sixty weekly cohorts averaged a $42.15 acquisition cost against the CFO's $45 ceiling, with a standard deviation of $7.80. Cohen's d is (42.15 minus 45) / 7.80 = -0.37, so the average sits about a third of a typical spread below the ceiling.

⚖️ How to Judge the Number: Zero means the average sits exactly on the benchmark; the sign only says which side. Read it next to the p-value and the gap in dollars: with a big enough sample, a d of 0.05 is significant but rarely worth acting on.

⚠️ Common Misread to Avoid: Treating a tiny p-value as proof of a big effect. With 20,000 rows, a d of 0.03 gives p below 0.001, yet the average has moved only 3% of one typical spread. Cohen's d does not grow with the sample; the p-value does.

📐 Technical Definition: This is the one-sample form the One-Sample Testing Lab prints. Two-group versions divide the gap between the group means by a pooled standard deviation instead. Cohen's (1988) bands are conventions for when no better yardstick exists, and a gap in dollars against a budget is better.

🎯 The Manager's Question: How large is the average within-unit change compared with the spread of individual changes?

📊 Worked Business Example: Suppose basket size rises by $2.15 on average across 20 stores and store change scores have SD = $1.27. Then d_z = 2.15 / 1.27 = 1.69.

⚖️ How to Judge the Number: Use 0.20 (small), 0.50 (medium), and 0.80 (large) alongside the raw mean change and baseline average.

⚠️ Common Misread to Avoid: Comparing d_z directly to an independent-groups Cohen’s d without checking the denominator. Because d_z divides by the SD of change scores, correlated pairs produce a larger d_z.

📐 Technical Definition: The paired t-statistic equals \(d_z \sqrt{n}\). When pre- and post-scores correlate positively, \(s_d\) is smaller than the raw group SD.

🎯 The Manager's Question: Is our conversion gain over the benchmark big, or does it only sound big?

📊 Worked Business Example: Aura Athletics converted 70 of 1,850 clicks (3.78%) against a 2.40% baseline. Cohen's h is about 0.08, which sits below Cohen's 0.2 "small" band, even though the conversion rate rose 58%.

⚖️ How to Judge the Number: Zero means the rate matches the benchmark; the bands are the same as for d. Near a low base rate those bands undersell real gains, so read h together with the lift in points, in percent and in dollars. Roll out the change when the dollar gain covers its cost, whatever band h falls in.

⚠️ Common Misread to Avoid: Reading h = 0.08 as "too small to matter". Near a 2.4% baseline h rates a large relative lift as small: Aura's 0.08 is a 58% lift that the scenario card values at +$74,044 a year. Judge the lift in dollars before dismissing it.

📐 Technical Definition: The arcsine square-root transform makes the sampling spread of a proportion about the same at every rate. So a gap of 0.1 in h is about equally easy to detect at 5% or at 50%. The One-Sample Testing Lab computes it from the sample and benchmark proportions.

🎯 The Manager's Question: Do customers who joined in different months hold on differently at the same age?

📊 Worked Business Example: At age 3, Parcel & Pantry's March and April 2024 cohorts keep 54% and 53% of their customers, against 73% to 80% for every other month. Those are the two months a 60%-off promotion ran.

⚖️ How to Judge the Number: Compare down a column, never across a row: only the same age is a fair comparison. The tool flags drift when cohorts differ by 10 points or more at the latest age at least half of them reach; here the six oldest differ by 33 points at month 19. Read it next to how each weak cohort was acquired.

⚠️ Common Misread to Avoid: Blaming the calendar month for a weak cohort. The usual culprit is how those customers were acquired: 89% of the March and April sign-ups here came through the 60%-off promotion, against under 6% in every other month.

📐 Technical Definition: Cells beyond the oldest age a cohort has reached are left blank rather than zero, so recent cohorts give shorter rows. The blank lower-right triangle is future that has not happened yet, not missing data.

🎯 The Manager's Question: What would {outcome} in the test markets have done without the {treatment}?

📊 Worked Business Example: Control stores rose $15K over the same months, so without the app the test stores would have gone from $420K to about $435K. Their actual $465K sits $30K above that counterfactual.

⚖️ How to Judge the Number: 0 means the market as a whole stood still. Add the control markets' change to the test markets' starting level to get the counterfactual, and read the DiD estimate against it. A common trend as big as the estimate means a before-and-after comparison would mostly measure the season.

⚠️ Common Misread to Avoid: Assuming the background trend is zero. In the app-launch data that credits the app with all $45K of the test stores' gain, when $15K would have arrived anyway. Seasonal and growing categories make this the most common overclaim.

📐 Technical Definition: The bracket is the common trend. It is a fair stand-in only if the test markets would have followed it, which is the parallel-trends assumption. The tool draws this counterfactual as the dashed line on the story chart.

🎯 The Manager's Question: How much of the variation in answers to this item do the retained factors explain?

📊 Worked Business Example: In the default one-factor solution, "Support green policies" has a communality of 0.42 and the sports item 0.00. The factor explains 42% of the variation in the first and none in the second.

⚖️ How to Judge the Number: 0 means the factors explain none of the item; 1 means all of it. The tool flags anything below 0.35, so 6 of the 8 default items are flagged. Read it next to the item's loading and wording: on a 5-point scale a low value prompts a judgement, not an automatic drop.

⚠️ Common Misread to Avoid: Dropping every item flagged for low communality. In the default survey that would leave two items, too few to measure anything. The flag asks whether the item belongs; an item that clearly does can stay at 0.30.

📐 Technical Definition: Rotation does not change it: the tool reports it from the unrotated solution, and after Promax it equals \(\operatorname{diag}(\Lambda \Phi \Lambda^{\top})_i\), where \(\Phi\) holds the factor correlations. Principal axis factoring starts each item at its squared multiple correlation and iterates until the communalities settle.

🎯 The Manager's Question: Does each factor hold its own items tightly and stay distinct from the other factors?

📊 Worked Business Example: In NovaBrew, once the three low-communality items are dropped, Social Image has AVE 0.65 and CR 0.85. Its square root of AVE, 0.80, is well above its largest correlation with another factor, 0.35, so it passes Fornell-Larcker.

⚖️ How to Judge the Number: An AVE of 0.50 is the line: below it, a factor explains less of its items than error does. A CR of 0.70 is the reliability line, read like alpha. For discriminant validity, read the square root of AVE against the factor's correlations with the other factors; it should beat every one. A failure means two factors overlap too much to report as separate ideas.

⚠️ Common Misread to Avoid: Treating CR as a second copy of alpha. CR is built from the factor loadings, so strong items count for more, and here it is the same number as that factor's omega. When CR and alpha differ, the factor's items have unequal loadings.

📐 Technical Definition: \(\text{CR} = (\sum \lambda_i)^2 / [(\sum \lambda_i)^2 + \sum (1 - \lambda_i^2)]\) over the same items. Fornell-Larcker passes when \(\sqrt{\text{AVE}}\) exceeds every absolute factor correlation \(\lvert\phi_{jk}\rvert\). The loadings come from the exploratory solution, so these are screens, and the tool does not compute HTMT, the stricter ratio the chapter recommends. Fornell-Larcker is a comparison rule, not a statistic of its own; it is listed here because the tool's Fornell-Larcker button opens this entry.

🎯 The Manager's Question: Which true values are still believable, given what this sample showed?

📊 Worked Business Example: Aura's 70 sales in 1,850 clicks give a 95% interval of 2.9% to 4.7%, in the card's small print under the one-sided bound. All of it sits above the 2.4% baseline, so the baseline is ruled out.

⚖️ How to Judge the Number: Check it against the no-change value: 0 for a difference, 1 for a ratio, the baseline for a rate. An interval that clears that value rules it out; one that straddles it cannot choose. Then read the width, which is the range of outcomes to cost out. Read it next to the p-value, which answers the same yes-or-no question but says nothing about size.

⚠️ Common Misread to Avoid: Reading a {level}% interval as a {level}% chance that the true value is inside this one. The {level}% describes the method: across many samples, intervals built this way catch the true value that often. This particular interval either holds it or does not, so read it as the values this data does not rule out.

📐 Technical Definition: It assumes the estimate's sampling distribution is close to the normal or t shape, which small samples and heavy skew can break. For a proportion the one-sample tool uses the Wald interval \(\hat{p} \pm z\sqrt{\hat{p}(1-\hat{p})/n}\), clipped to 0 and 1; for a mean it uses t. A one-tailed test also shows a one-sided bound: Aura's rate is at least 3.05%. Intervals for ratios, such as an odds ratio, are built on the log scale, so they sit lopsided around the estimate.

🎯 The Manager's Question: Which categories does the model mix up, and how often?

📊 Worked Business Example: Of Kinoscope’s 600 held-back subscribers, 221 really chose Standard. The network put 103 of them on Standard and 99 on Basic with Ads, so it mistakes Standard for the cheaper plan almost as often as it gets it right.

⚖️ How to Judge the Number: Read across a row: the row is what customers really chose, and each cell is what they were predicted to choose. The diagonal holds the right answers, and a row’s diagonal share is that category’s recall (103 of 221, 47%, for Standard). Read it next to accuracy (59% here) and the 47% from always guessing the most common plan.

⚠️ Common Misread to Avoid: Reading down a column as if it were a row. A column is everyone predicted to be one category, which answers a different question: of the 324 Kinoscope subscribers predicted Basic with Ads, 202 really chose it.

📐 Technical Definition: A table of counts with rows for the actual category and columns for the predicted one. In the Neural Network Lab the prediction is the category with the highest predicted probability, over the held-back rows. In logistic regression it is the 2 × 2 table at the cutoff set on the slider.

🎯 The Manager's Question: Which product features swing a typical customer's choices the most across the tested levels?

📊 Worked Business Example: Suppose Heat spans 4 utility points and Brand spans 2 for every shopper, even though half want Hot and half want Mild. Each person gives Heat 4 / 6 = 66.7% importance, so the sample mean stays 66.7%.

⚖️ How to Judge the Number: Importances sum to 100%, so an even split across five attributes is 20% each. Read this column next to the average-utilities column: a gap of 10 points or more flags opposing customer tastes.

⚠️ Common Misread to Avoid: Reading 40% importance as "this feature explains 40% of choices". It is each person's share of the tested utility ranges, averaged across respondents, so testing a wider price range raises price's share. It also says nothing about which level customers prefer.

📐 Technical Definition: Each respondent's attribute range is normalized to 100% before averaging across the sample. Because ranges are non-negative for every person, opposing preferences do not cancel out. Both Basic Conjoint and Advanced Conjoint report this metric.

🎯 The Manager's Question: Which product features move the sample-average utility the most when all respondents are pooled together?

📊 Worked Business Example: Suppose half the sample scores Hot at +2 and half at -2, while everyone scores Brand Y at +1. Averaging first cancels Heat to 0% and gives Brand 100%, hiding that every person cared more about Heat.

⚖️ How to Judge the Number: Importances sum to 100%, so five attributes split evenly at 20% each. Read this column next to mean individual importance: a drop of 10 points or more signals opposing tastes that cancel in the average.

⚠️ Common Misread to Avoid: Treating a low score here, such as 9% on Offline Downloads, as proof that customers do not care about that feature. Because utilities are averaged across respondents first, equal-sized segments with opposite preferences cancel toward zero. Check mean individual importance and the utility standard deviations before cutting a feature.

📐 Technical Definition: For each attribute level, utilities are averaged across all respondents first. Each attribute's range is then computed from those sample-mean utilities and divided by the sum of sample-mean ranges. Unlike mean individual importance, opposing segment preferences cancel before the range is taken.

🎯 The Manager's Question: What share of the variation in {outcome} is accounted for by the active text constructs?

📊 Worked Business Example: In Canyon Creek Bistro, four operational constructs account for 62.4% of the variation in star ratings across 150 reviews. The remaining 37.6% reflects unmodeled topics and personal rating habits.

⚖️ How to Judge the Number: In customer review data, values from 0.25 to 0.60 indicate strong construct coverage, read alongside Adjusted R-squared and the omnibus F-test.

⚠️ Common Misread to Avoid: Expecting review models to reach 0.90 like aggregate sales time series. Individual ratings contain personal baseline noise, and low values mean key topics are missing from your construct list.

📐 Technical Definition: Adding extra constructs mechanically raises sample R-squared. Check Adjusted R-squared to verify that a new construct chip genuinely improves fit.

🎯 The Manager's Question: How much does {outcome} shift when a review moves from zero to full construct entailment?

📊 Worked Business Example: In Canyon Creek Bistro, Service Delay has a slope of -1.68 on star_rating. Going from 0.00 to 1.00 on Service Delay lowers the predicted rating by 1.68 stars, holding other constructs constant.

⚖️ How to Judge the Number: A slope of 0.00 means no partial effect on {outcome}. Read the slope next to its 95% confidence interval and construct prevalence.

⚠️ Common Misread to Avoid: Multiplying the slope by word count instead of the 0-to-1 probability range. Because each construct probability is bounded between 0 and 1, the slope is the full-range shift across that interval.

📐 Technical Definition: Ordinary least squares estimates each partial slope holding the other constructs constant. This separates co-occurring complaints inside mixed reviews.

🎯 The Manager's Question: How strongly does this construct probability correlate with {outcome} on its own?

📊 Worked Business Example: In Canyon Creek Bistro, Service Delay correlates with star_rating at r = -0.66 before controlling for food temperature or price.

⚖️ How to Judge the Number: It runs from -1.00 to +1.00. Read it next to the partial OLS slope to see whether co-occurring complaints inflated the simple correlation.

⚠️ Common Misread to Avoid: Treating a strong simple correlation as proof that the construct drives {outcome} independently. Frustrated reviewers often mention secondary gripes alongside their main complaint.

📐 Technical Definition: Compare simple r_k against partial slope b_k. A large r_k with a near-zero b_k means another correlated construct accounts for the rating drop.

🎯 The Manager's Question: At what entailment probability do we count a construct as present in a review?

📊 Worked Business Example: At a 0.50 cutoff, 54 of 150 reviews mention Service Delay (36%). Raising it to 0.70 keeps 39 explicit hits (26%).

⚖️ How to Judge the Number: The default 0.50 marks where entailment beats contradiction. Read prevalence across cutoffs from 0.20 to 0.80 alongside mean probability.

⚠️ Common Misread to Avoid: Thinking the presence cutoff changes the OLS regression slopes. The regression always fits on continuous probabilities, while tau only governs binary prevalence counts.

📐 Technical Definition: Lowering tau increases recall for subtle mentions. Raising tau increases precision for unambiguous mentions.

🎯 The Manager's Question: What share of reviews in the sample express this construct above the probability cutoff?

📊 Worked Business Example: Across 150 Canyon Creek Bistro reviews, 54 reviews score at or above 0.50 on Service Delay. That gives a prevalence of 36.0%.

⚖️ How to Judge the Number: Compare prevalence from 0% to 100% across constructs alongside each construct mean score and regression slope.

⚠️ Common Misread to Avoid: Assuming the highest-prevalence construct is the biggest driver of ratings. A construct mentioned in 60% of reviews may leave star ratings unchanged, while a 20% complaint drives 1-star reviews.

📐 Technical Definition: Lowering the probability threshold from 0.50 to 0.35 increases sensitivity to subtle mentions at the cost of more false positives.

🎯 The Manager's Question: Do two or more construct hypotheses overlap so closely that OLS cannot separate their slopes?

📊 Worked Business Example: In Canyon Creek Bistro, the four default constructs have VIF values near 1.25. Adding a duplicate chip like "Slow Table Seating" next to "Service Delay" pushes VIF above 5.0.

⚖️ How to Judge the Number: A VIF of 1.00 means zero overlap. Values at or above 5.00 warn of severe overlap, read alongside the standard error.

⚠️ Common Misread to Avoid: Ignoring high VIF when switching to Single-Label mutually exclusive mode. Forcing construct probabilities to sum to 1.0 creates structural collinearity across the predictors.

📐 Technical Definition: The square root of VIF measures standard error inflation. It shows how many times wider the standard error is than with orthogonal constructs.

🎯 The Manager's Question: Which construct accounts for the largest share of outcome swings across the whole corpus?

📊 Worked Business Example: Service Delay has a standardized beta of -0.49 in Canyon Creek Bistro, compared with -0.38 for Cold Food. That ranks Service Delay as the top corpus-wide driver of star-rating swings.

⚖️ How to Judge the Number: Values beyond 0.20 or -0.20 mark substantial drivers in review regressions, read alongside the unstandardized slope and prevalence.

⚠️ Common Misread to Avoid: Confusing standardized beta with the per-review severity of a defect. A rare hardware failure in 4% of reviews can have a steep unstandardized slope but a modest standardized beta.

📐 Technical Definition: Rescaling by standard deviations puts every construct on a common unitless scale. It shows how many SDs {outcome} moves per one-SD rise in construct probability.

🎯 The Manager's Question: How many people fall into each combination of group and outcome?

📊 Worked Business Example: Odds-O-Matic starts with 1,000 people per segment: 120 in Segment A convert and 880 do not, against 40 and 960 in Segment B. Every probability, odds and ratio on the page comes from those four counts.

⚖️ How to Judge the Number: No cell should be tiny: with fewer than about 5 people in a cell, the ratios swing with one or two people. A cell of 0 makes the odds of that group 0 or infinite, so the odds ratio cannot be read. Read it with the row totals, since each group's rate is its own row divided by its own total.

⚠️ Common Misread to Avoid: Thinking unequal group sizes bias the comparison. If Segment A had 2,000 people and Segment B had 500, each rate is still its own row divided by its own total. What matters is how many people each row holds, because a small row gives a noisy rate.

📐 Technical Definition: Rows are the groups and columns the outcome, so each row total is a group's size and each column total is how many had or lacked the outcome. Group A's rate is \(a/(a+b)\), its odds are \(a/b\), and the odds ratio is \(ad/bc\). The same layout holds any two yes/no questions, such as a condition against a test result.

🎯 The Manager's Question: How many people fall into each combination of group and outcome?

📊 Worked Business Example: Odds-O-Matic starts with 1,000 people per segment: 120 in Segment A convert and 880 do not, against 40 and 960 in Segment B. Every probability, odds and ratio on the page comes from those four counts.

⚖️ How to Judge the Number: No cell should be tiny: with fewer than about 5 people in a cell, the ratios swing with one or two people. A cell of 0 makes the odds of that group 0 or infinite, so the odds ratio cannot be read. Read it with the row totals, since each group's rate is its own row divided by its own total.

⚠️ Common Misread to Avoid: Thinking unequal group sizes bias the comparison. If Segment A had 2,000 people and Segment B had 500, each rate is still its own row divided by its own total. What matters is how many people each row holds, because a small row gives a noisy rate.

📐 Technical Definition: Rows are the groups and columns the outcome, so each row total is a group's size and each column total is how many had or lacked the outcome. Group A's rate is \(a/(a+b)\), its odds are \(a/b\), and the odds ratio is \(ad/bc\). The same layout holds any two yes/no questions, such as a condition against a test result.

🎯 The Manager's Question: How many visitors took the action you wanted?

📊 Worked Business Example: In the default Nordvik race, 20,000 visitors give the oracle 832 conversions, Thompson sampling 818, the A/B test 788 and epsilon-greedy 759.

⚖️ How to Judge the Number: Read it against the oracle line, the most any approach could earn from these same visitors. The gap is the regret, and 0 means a perfect match. Then read it next to what each approach learned, because the year after the test usually earns more than the test.

⚠️ Common Misread to Avoid: Reading more conversions during the race as the better approach. The race lasts weeks; what you do for the next year with what you learned usually matters more. In this single race the A/B test's 30 fewer conversions bought an interval on the losing hero about a third narrower, 1.05 against 1.61 points.

📐 Technical Definition: A visitor converts on an option if their draw falls below that option’s true rate. Every contestant faces the same draws, so differences in conversions come only from which option each one served.

🎯 The Manager's Question: Is one customer or campaign pulling the whole fitted line toward itself?

📊 Worked Business Example: In the 60 ad campaigns the largest Cook's distance is 0.33, for the $6.6k campaign that missed by $22k. In the influencer preset one celebrity partnership scores 181, so dropping it would change the line completely.

⚖️ How to Judge the Number: Zero means dropping the case would not move the fit at all. Above 0.5 deserves scrutiny, and above 1 means one case is steering the fit on its own. The looser 4/n screen flags cases worth a second look: 0.067 with 60 rows. Read it next to the leverage and the standardized residual, which show whether the pull comes from an unusual predictor value, a big miss, or both.

⚠️ Common Misread to Avoid: Treating a large value as a reason to delete the case. It says the case matters, not that it is wrong: check it for an entry error, then report the fit with and without it. The celebrity partnership in the influencer preset is real, just unlike the other 50.

📐 Technical Definition: The display form shows that influence needs both a large miss and high leverage. It equals the deletion form \(D_i = \sum_j(\hat{y}_j - \hat{y}_{j(i)})^2/(k s^2)\), where \(\hat{y}_{j(i)}\) is the fitted value for case j with case i left out and s is the residual standard error. The 0.5 and 1 cut-offs are rules of thumb, not tests.

🎯 The Manager's Question: Which cases deserve a second look because dropping any one of them would shift the fit noticeably?

📊 Worked Business Example: With 60 ad campaigns the cut-off is 4/60, or 0.067, and 4 campaigns cross it, the largest at 0.33. None comes near the 0.5 and 1 lines that mark a dominant case.

⚖️ How to Judge the Number: It is a screening cut-off that shrinks as the sample grows: 0.04 with 100 rows, 0.004 with 1,000. Read it beside the 0.5 and 1 thresholds, which flag only dominant cases, and next to the leverage of each flagged row.

⚠️ Common Misread to Avoid: Using it as a deletion rule. In a large sample many cases cross it by chance. Check each flagged row, then report results with and without the ones that matter.

📐 Technical Definition: It is a rule of thumb, often credited to Bollen and Jackman (1990), not a test. So the share of cases crossing it does not by itself say the model is wrong.

🎯 The Manager's Question: Is one customer or campaign pulling the whole fitted line toward itself?

📊 Worked Business Example: In the 60 ad campaigns the largest Cook's distance is 0.33, for the $6.6k campaign that missed by $22k. In the influencer preset one celebrity partnership scores 181, so dropping it would change the line completely.

⚖️ How to Judge the Number: Zero means dropping the case would not move the fit at all. Above 0.5 deserves scrutiny, and above 1 means one case is steering the fit on its own. The looser 4/n screen flags cases worth a second look: 0.067 with 60 rows. Read it next to the leverage and the standardized residual, which show whether the pull comes from an unusual predictor value, a big miss, or both.

⚠️ Common Misread to Avoid: Treating a large value as a reason to delete the case. It says the case matters, not that it is wrong: check it for an entry error, then report the fit with and without it. The celebrity partnership in the influencer preset is real, just unlike the other 50.

📐 Technical Definition: The display form shows that influence needs both a large miss and high leverage. It equals the deletion form \(D_i = \sum_j(\hat{y}_j - \hat{y}_{j(i)})^2/(k s^2)\), where \(\hat{y}_{j(i)}\) is the fitted value for case j with case i left out and s is the residual standard error. The 0.5 and 1 cut-offs are rules of thumb, not tests.

🎯 The Manager's Question: Is one customer or campaign pulling the whole fitted line toward itself?

📊 Worked Business Example: In the 60 ad campaigns the largest Cook's distance is 0.33, for the $6.6k campaign that missed by $22k. In the influencer preset one celebrity partnership scores 181, so dropping it would change the line completely.

⚖️ How to Judge the Number: Zero means dropping the case would not move the fit at all. Above 0.5 deserves scrutiny, and above 1 means one case is steering the fit on its own. The looser 4/n screen flags cases worth a second look: 0.067 with 60 rows. Read it next to the leverage and the standardized residual, which show whether the pull comes from an unusual predictor value, a big miss, or both.

⚠️ Common Misread to Avoid: Treating a large value as a reason to delete the case. It says the case matters, not that it is wrong: check it for an entry error, then report the fit with and without it. The celebrity partnership in the influencer preset is real, just unlike the other 50.

📐 Technical Definition: The display form shows that influence needs both a large miss and high leverage. It equals the deletion form \(D_i = \sum_j(\hat{y}_j - \hat{y}_{j(i)})^2/(k s^2)\), where \(\hat{y}_{j(i)}\) is the fitted value for case j with case i left out and s is the residual standard error. The 0.5 and 1 cut-offs are rules of thumb, not tests.

🎯 The Manager's Question: Which cases deserve a second look because dropping any one of them would shift the fit noticeably?

📊 Worked Business Example: With 60 ad campaigns the cut-off is 4/60, or 0.067, and 4 campaigns cross it, the largest at 0.33. None comes near the 0.5 and 1 lines that mark a dominant case.

⚖️ How to Judge the Number: It is a screening cut-off that shrinks as the sample grows: 0.04 with 100 rows, 0.004 with 1,000. Read it beside the 0.5 and 1 thresholds, which flag only dominant cases, and next to the leverage of each flagged row.

⚠️ Common Misread to Avoid: Using it as a deletion rule. In a large sample many cases cross it by chance. Check each flagged row, then report results with and without the ones that matter.

📐 Technical Definition: It is a rule of thumb, often credited to Bollen and Jackman (1990), not a test. So the share of cases crossing it does not by itself say the model is wrong.

🎯 The Manager's Question: Across many repeats of this race, how often does this approach end on the truly best option?

📊 Worked Business Example: With the Monte Carlo view set to 500 Nordvik races, the A/B test ships the better hero 83% of the time and Thompson sampling 99.6%. The default 100 races show 88% and 99%.

⚖️ How to Judge the Number: Read it against 80%, the power the A/B test is sized for, and against a blind guess, 50% with two options. Then read it next to mean regret. At 500 races the A/B test picks right 83% of the time and gives up 71 conversions; Thompson sampling, 99.6% and 18.

⚠️ Common Misread to Avoid: Reading one race as a verdict on the approach. A single race either picked right or did not; the 83% only shows up across 500 of them. Run the Monte Carlo view before judging an approach.

📐 Technical Definition: The share of simulated races whose final choice is a truly best option. The A/B test's choice is what it ships; a bandit's is the option with the highest estimated rate at the end. When options are tied, any pick counts as right, so the rate says nothing there.

🎯 The Manager's Question: Does this item rise and fall with the rest of the scale?

📊 Worked Business Example: In the default survey, "Support green policies" correlates 0.53 with the total of the other seven items. "Watch sports on weekend" correlates 0.02: people who care about the environment are no more or less likely to watch sports.

⚖️ How to Judge the Number: 0 means the item is unrelated to the rest of the scale. The tool flags anything below 0.30 as weak; the chapter's stricter screen is 0.40. A negative value usually means the item is worded the other way round. Read it next to alpha if item deleted: a weak item normally raises alpha when it goes.

⚠️ Common Misread to Avoid: Deleting an item because its correlation is negative. Negative usually means it is worded in reverse, such as "The setup was confusing" among positive items. Tick it for reverse coding first; it then often correlates as well as the rest.

📐 Technical Definition: "Corrected" means the item is left out of the total it is correlated with; leaving it in would inflate the correlation, most of all on short scales. It is a Pearson correlation on complete responses and matches the Python reference to six decimals.

🎯 The Manager's Question: How does the model turn predictor values and exposure time into an expected event count?

📊 Worked Business Example: In Mosaic Analytics, an account with 10 active months and a log-rate score of -0.35 has an expected rate of exp(-0.35) = 0.705 tickets per month. Over 10 active months, the expected count is 10 * 0.705 = 7.05 tickets.

⚖️ How to Judge the Number: A log-rate score of 0.00 corresponds to 1.00 expected event per unit of exposure. Read the score alongside the exponentiated incidence rate ratios and the exposure window to translate log-units into real counts.

⚠️ Common Misread to Avoid: Adding coefficients directly to the expected count as if the model were linear. Because the link function is logarithmic, adding to the linear score multiplies the expected count rather than shifting it by a flat amount.

📐 Technical Definition: On the log-link scale, \(\ln(\hat{\mu}_i) = \ln(E_i) + \hat{\beta}_0 + \sum_j \hat{\beta}_j X_{ij}\). Exponentiating guarantees strictly positive expected counts \(\hat{\mu}_i > 0\) and makes every exponentiated slope \(e^{\hat{\beta}_j}\) a multiplicative rate ratio.

🎯 The Manager's Question: How much ad spend does each credited purchase cost us?

📊 Worked Business Example: Plan A's $10,000 buys 2,000,000 impressions, 10,000 clicks and 200 purchases, so CPA is $10,000 ÷ 200 = $50. Halve the purchase rate to 1% and CPA doubles to $100.

⚖️ How to Judge the Number: Judge it against what a purchase is worth: a $50 CPA pays off only if each purchase earns more than $50 in gross margin. With zero purchases it cannot be computed. Read it next to the click rate and the purchase rate, which show where in the funnel the cost comes from.

⚠️ Common Misread to Avoid: Reading a $50 CPA as $50 per sale the ads created. It counts credited purchases, and some of those buyers would have bought anyway; only a holdout test measures the extra sales.

📐 Technical Definition: The tool computes it from unrounded expected counts and shows no value, rather than an infinite one, when expected purchases are 0. Spend is media only; creative, fees and product margin are left out.

🎯 The Manager's Question: How much does it cost to show the ad 1,000 times, and how many showings will the budget buy?

📊 Worked Business Example: Plan A pays a $5 CPM, so its $10,000 budget buys $10,000 ÷ $5 × 1,000 = 2,000,000 impressions. One single impression costs $5 ÷ 1,000 = $0.005.

⚖️ How to Judge the Number: A CPM is a price, so judge it against another plan's CPM for the same ad format and the same impression rule. Doubling it halves what a budget buys: $10,000 at a $10 CPM buys 1,000,000 impressions, not 2,000,000. Read it together with the click rate and the cost per purchase, since a cheap impression nobody clicks can cost more per sale.

⚠️ Common Misread to Avoid: Reading a $5 CPM as $5 for each ad shown. It is $5 per 1,000 impressions, so one impression costs half a cent, and the price says nothing about clicks or purchases.

📐 Technical Definition: The simulator uses it two ways: in the auction every bid is a CPM, and in the campaign plan impressions = budget ÷ CPM × 1,000. Spend is media only; creative, service fees and taxes are left out of both plans.

🎯 The Manager's Question: Do these survey items agree closely enough to add them up into one score?

📊 Worked Business Example: In the default Environmental Consciousness survey, 120 respondents answer 8 items and alpha is 0.69, just under the usual 0.70 line. Untick the sports item, which has nothing to do with the environment, and alpha rises to 0.76.

⚖️ How to Judge the Number: 0 means the items share nothing; 1 means they agree perfectly. Above 0.95 the items are probably asking the same question twice. Read it next to its 95% interval: the default 0.69 runs from 0.60 to 0.77, so it could be questionable or acceptable. Clear 0.70 before you average the items into one score and report it.

⚠️ Common Misread to Avoid: Reading a high alpha as a sign that the items measure one single idea. Two related ideas mixed together can still reach 0.80 or more, because alpha rises with the number of items. Check the factor count from parallel analysis before treating the score as one idea.

📐 Technical Definition: Alpha assumes every item reflects the idea equally well (tau-equivalence). When the loadings differ it understates reliability, and omega is the better estimate. The tool uses complete responses only, sample variances with \(n - 1\), and the Feldt F-based 95% interval. It matches an independent Python reference (scipy and factor_analyzer) to six decimals.

🎯 The Manager's Question: Do these survey items agree closely enough to add them up into one score?

📊 Worked Business Example: In the default Environmental Consciousness survey, 120 respondents answer 8 items and alpha is 0.69, just under the usual 0.70 line. Untick the sports item, which has nothing to do with the environment, and alpha rises to 0.76.

⚖️ How to Judge the Number: 0 means the items share nothing; 1 means they agree perfectly. Above 0.95 the items are probably asking the same question twice. Read it next to its 95% interval: the default 0.69 runs from 0.60 to 0.77, so it could be questionable or acceptable. Clear 0.70 before you average the items into one score and report it.

⚠️ Common Misread to Avoid: Reading a high alpha as a sign that the items measure one single idea. Two related ideas mixed together can still reach 0.80 or more, because alpha rises with the number of items. Check the factor count from parallel analysis before treating the score as one idea.

📐 Technical Definition: Alpha assumes every item reflects the idea equally well (tau-equivalence). When the loadings differ it understates reliability, and omega is the better estimate. The tool uses complete responses only, sample variances with \(n - 1\), and the Feldt F-based 95% interval. It matches an independent Python reference (scipy and factor_analyzer) to six decimals.

🎯 The Manager's Question: Does this survey item measure two of the ideas at once?

📊 Worked Business Example: In the PureGlow survey, "Natural ingredients that actually work" loads 0.52 on Authenticity and 0.45 on Efficacy. Its answers say something about both ideas, so it cannot cleanly score either one.

⚖️ How to Judge the Number: A clean item loads on one factor and near 0 on the rest. The tool flags a cross-loader when the second loading is 0.30 or more and within 0.20 of the first. Read it next to the item's wording, then drop it or rewrite it to ask about one idea only.

⚠️ Common Misread to Avoid: Thinking an item that loads on two factors is twice as useful. It blurs the line between the two scores and inflates their correlation. PureGlow's two cross-loaders are among the four items to drop on the way to 15 clean ones.

📐 Technical Definition: The tool applies the rule to the Promax pattern matrix, so each loading is a unique contribution with the other factors held fixed. The same item also correlates strongly with both factors in the structure matrix: 0.67 and 0.61 for the PureGlow item.

🎯 The Manager's Question: How different are customers in their usual chance of buying, before any habit?

📊 Worked Business Example: Model C puts the middle 80% of Thistledown shoppers between a 1.8% and a 43.9% chance of buying cold brew in a typical week. A shopper at the top is about 25 times as likely to buy as one at the bottom.

⚖️ How to Judge the Number: A range of zero width, as model A always shows, treats every customer as the same. Saltmarsh's 10.6% to 12.3% is narrow, so loyalty explains almost none of its habit. Thistledown's top shopper is about 25 times as likely to buy as its bottom one, so loyalty can explain most of it. Read it next to the habit effect under models A to C; a wide spread plus a habit that shrinks from A to C argues against paying for trials to build a habit.

⚠️ Common Misread to Avoid: Planning around the "average customer" when this range is wide. The median Thistledown shopper buys in about 10% of weeks, yet most shoppers sit well above or well below that. A campaign tuned to the median fits almost nobody.

📐 Technical Definition: The range shown is the 10th to 90th percentile of \(p_i\) across customers. Under model B it covers the known part only, so it is narrower (10.5% to 26.7% in Thistledown). Under model C it adds \(u_i\), which assumes the unexplained part follows a bell curve on the log-odds scale.

🎯 The Manager's Question: Which broad bucket did analytics file this visit in, and can I trust that bucket?

📊 Worked Business Example: In the lab, a link tagged utm_source=instagram and utm_medium=cpc is filed as Paid Social. The same link with utm_medium=social is filed as Organic Social, so $14,299 of paid revenue shows up as free traffic.

⚖️ How to Judge the Number: The group comes from the tags and the referrer, not from where the money went. A correct plan files 100% of sessions in the group that earned them; the lab calls that share coverage. Read it next to the Session source / medium report: a paid source sitting in Organic, Direct or Unassigned means the label is wrong.

⚠️ Common Misread to Avoid: Reading "Direct $61,986" as customers who typed the address in. Direct is where visits with no referrer and no tags land, which includes every click from an email app. In the lab, untagged email adds $49,995 to Direct and leaves Email at $0.

📐 Technical Definition: The lab applies GA4's manual-traffic rules in order: the medium decides paid against unpaid, a recognised social or search source decides the family, and anything the rules cannot place falls to Unassigned. The rules live in utm_rules.js and follow Google's published channel definitions.

🎯 The Manager's Question: How many of the numbers in this test were free to vary once the totals were fixed?

📊 Worked Business Example: Zenith tested 5 flavors with the total fixed at 1,250, so once 4 flavor counts are known the fifth is forced. The test has 5 minus 1, or 4, degrees of freedom.

⚖️ How to Judge the Number: It sets which reference curve the statistic is read against, so it has no good or bad value. For a goodness-of-fit test with k categories it is k minus 1; for a one-sample t-test on n rows it is n minus 1. Read the statistic against the cut-off for its df: at the 0.05 level a chi-square needs 3.84 with 1 df but 9.49 with 4.

⚠️ Common Misread to Avoid: Reading df as the sample size. Zenith had 1,250 testers but only 4 degrees of freedom, because df counts the categories that can vary, not the people. Adding testers makes the test more sensitive without changing its df.

📐 Technical Definition: In general df is the number of values minus the number of constraints or parameters estimated from them. A goodness-of-fit test loses one for the fixed total, and one more for each benchmark parameter estimated from the same data. A one-sample t-test has \(n - 1\), where n is the number of rows.

🎯 The Manager's Question: Did scrubbing low-quality responses accidentally shift the demographic makeup of your retained sample?

📊 Worked Business Example: In GlowRoute's TikTok giveaway survey, scrubbing removes hurried mobile clickers. That drops the Mobile share by 9.14 percentage points (chi-square p < 0.001) and triggers a drift warning.

⚖️ How to Judge the Number: 0.00 percentage points means sample composition is unchanged. Shifts under 3.0 percentage points with chi-square p >= 0.05 indicate Composition Holds, read alongside Cramér's V.

⚠️ Common Misread to Avoid: Assuming data cleaning never biases sample representativeness. When careless responding clusters in one device or age cohort, scrubbing alters subgroup shares and requires disclosure or post-stratification weighting.

📐 Technical Definition: A contingency table chi-square test compares retained versus scrubbed counts across all \(C\) demographic categories. Cramér's \(V\) and Total Variation Distance quantify the magnitude of composition shift.

🎯 The Manager's Question: How much of the mismatch between predicted chances and what customers did do the predictors remove?

📊 Worked Business Example: In the influencer preset the null deviance is 1,380.8 and the residual deviance 1,064.3. The predictors remove 316.5, which is the model chi-square on 7 degrees of freedom.

⚖️ How to Judge the Number: The null deviance gives every row the overall rate, 46.3% here; the residual deviance uses the predictors. The drop between them is tested against a chi-square with one degree of freedom per added term, which is the likelihood ratio test. Read it next to McFadden R², which is 1 minus residual over null, 0.229 here.

⚠️ Common Misread to Avoid: Reading it as a count of wrong predictions or a percentage. It is a sum over rows, so it grows with the sample: 1,064 on 1,000 rows is not a bad fit by itself. Compare deviances only between models fitted to the same rows.

📐 Technical Definition: For 0/1 rows the saturated model fits perfectly, so the deviance equals -2 times the log-likelihood. The null deviance sets every \(\hat{p}_i\) to the overall rate. The tool reports both on the fitting rows.

🎯 The Manager's Question: How many of the numbers in this test were free to vary once the totals were fixed?

📊 Worked Business Example: Zenith tested 5 flavors with the total fixed at 1,250, so once 4 flavor counts are known the fifth is forced. The test has 5 minus 1, or 4, degrees of freedom.

⚖️ How to Judge the Number: It sets which reference curve the statistic is read against, so it has no good or bad value. For a goodness-of-fit test with k categories it is k minus 1; for a one-sample t-test on n rows it is n minus 1. Read the statistic against the cut-off for its df: at the 0.05 level a chi-square needs 3.84 with 1 df but 9.49 with 4.

⚠️ Common Misread to Avoid: Reading df as the sample size. Zenith had 1,250 testers but only 4 degrees of freedom, because df counts the categories that can vary, not the people. Adding testers makes the test more sensitive without changing its df.

📐 Technical Definition: In general df is the number of values minus the number of constraints or parameters estimated from them. A goodness-of-fit test loses one for the fixed total, and one more for each benchmark parameter estimated from the same data. A one-sample t-test has \(n - 1\), where n is the number of rows.

🎯 The Manager's Question: How much did the {treatment} change {outcome} beyond what the control markets say would have happened anyway?

📊 Worked Business Example: In the default app-launch data, test stores rose from $420K to $465K while control stores rose from $400K to $415K. The DiD estimate is $45K minus $15K, so the app added about $30K per store.

⚖️ How to Judge the Number: 0 means the test markets moved exactly as the control markets did. Weigh the estimate against what the {treatment} cost per market, and read it next to its interval: if the interval includes 0, the data cannot rule out no effect. It counts as a causal lift only if the pre-trend check and the placebo pass.

⚠️ Common Misread to Avoid: Reading the $30K as the app's effect, full stop. It is the effect only if test and control markets would have moved together without the launch, which is the parallel-trends assumption. Check the pre-trend test and the placebo before quoting it.

📐 Technical Definition: It assumes parallel trends and no spillover from test to control markets. On panel data the tool fits two-way fixed effects, \(y_{it} = \alpha_i + \lambda_t + \delta D_{it} + \varepsilon_{it}\), which gives the same number as the 2×2 formula on a balanced panel. It matches statsmodels to 1e-8 on every preset.

🎯 The Manager's Question: What range of lifts fits this data, and does it rule out no lift at all?

📊 Worked Business Example: In the Hearthstone Coffee preset the DiD estimate is $3,230 a week, with an interval of $2,914 to $3,546 from 20 markets. The default app-launch data is noise-free, so its interval shrinks to the point.

⚖️ How to Judge the Number: If the interval includes 0, the data cannot rule out no effect or a loss. Compare its lower end against the break-even lift: when even $2,914 a week covers the cost, the lift pays at the low end too. Read it next to the number of markets, because the interval widens fast when there are few.

⚠️ Common Misread to Avoid: Reading it as a {level}% chance that the true lift lies inside. The method captures the true lift in {level}% of repeated studies, and only if parallel trends hold. A tight interval around a biased estimate is still wrong.

📐 Technical Definition: Clustering by market lets each market's errors be correlated over time, but assumes markets are independent of one another. The tool uses the CR1 small-sample correction \(\tfrac{G}{G-1}\tfrac{N-1}{N-k}\) and matches statsmodels to 1e-4.

🎯 The Manager's Question: How much does {predictor} still move {outcome} by routes that bypass the mediator?

📊 Worked Business Example: In the Artisan Bean case, price still lifts willingness to pay by c′ = 0.31 standard deviations with perceived quality held fixed. That is more than the 0.26 that runs through quality, so a higher price also works as an anchor on its own.

⚖️ How to Judge the Number: Zero means everything {predictor} does runs through the mediator. Read it next to the indirect effect. Both of the same sign and clear of zero is partial mediation, as in the coffee case. A direct effect of the opposite sign is suppression, and then no share of the total should be reported. Its own interval against 0, or its p-value against {alpha}, is the test.

⚠️ Common Misread to Avoid: A direct effect that is not significant is read as proof that the mediator is the only route. Usually the sample is too small to see a second route. In the full-mediation case c′ = 0.08, but its interval runs from about -0.14 to 0.30.

📐 Technical Definition: Assumes the outcome equation is correctly specified. The tool fits it by ordinary least squares, as in PROCESS model 4. With no missing data, \(c = c' + ab\) exactly, so the direct, indirect and total effects are one split of the same total, not three separate findings. Adding or dropping a control changes it, because it is a coefficient in a model.

🎯 The Manager's Question: How much of this Direct row is customers typing the address, and how much is unlabelled traffic?

📊 Worked Business Example: With every lab source tagged, (direct) / (none) holds 4,200 sessions and $11,991. Untag the email links and it holds 13,400 sessions and $61,986, because mail apps send no referrer.

⚖️ How to Judge the Number: Compare the row with what you know people type in: this brand's baseline is about 4,200 sessions a month. Read it next to the Email row, and next to any channel that shows $0 against money you spent. A Direct row that jumps when a campaign launches is a labelling failure, not a surge of loyal customers.

⚠️ Common Misread to Avoid: Reading a large Direct row as proof that brand awareness is strong. Direct is the bucket for visits that arrive with nothing to read, which includes untagged email, app links and messaging. In the lab, $49,995 of email revenue sits there when the sends go out untagged.

📐 Technical Definition: A session with no referrer and no utm parameters is recorded with source (direct) and medium (none). Email clients, SMS and most apps strip the referrer, so their untagged clicks are indistinguishable from a typed address.

🎯 The Manager's Question: How strongly do we shrink rare, low-count words toward zero in the lexical contrast?

📊 Worked Business Example: Setting alpha_0 to 50 adds 50 pseudocounts across the vocabulary. Raising alpha_0 to 150 shrinks a 3-versus-0 word below z = 1.96.

⚖️ How to Judge the Number: The default 50 balances sensitivity and stability. Compare z-scores across priors from 10 to 200 alongside total word frequency.

⚠️ Common Misread to Avoid: Setting alpha_0 to its minimum and treating a 2-versus-0 typo as a major discovery. Low prior strength lets sparse noise masquerade as distinctive vocabulary.

📐 Technical Definition: In Monroe, Colaresi, and Quinn (2008), alpha_0 represents the equivalent sample size of a neutral background corpus. It pools with each group before computing log-odds.

🎯 The Manager's Question: How far below the $50 list price does a promo code sell the product?

📊 Worked Business Example: At the default 40% off the deal price is $30 and the run settles at an 8.3% margin. At 10% off the deal price is $45 and the margin settles at 40.6%.

⚖️ How to Judge the Number: Deal price = $50 × (1 − d) against a $27.50 unit cost, so 45% off is break-even and anything deeper loses money on every deal sale. The pull on exposed shoppers grows with depth up to 40% off. Read it next to Realized Gross Margin and the average fair price, since a deeper code also drags down what neighbors think is fair.

⚠️ Common Misread to Avoid: Assuming a deeper discount only costs margin on the shoppers who redeem it. Every deal sale also lowers what neighbors consider a fair price. At 40% off the average fair price falls from $45.06 to $31.74.

📐 Technical Definition: An exposed full-price shopper starts using codes with a yearly chance of \(0.3 \times \min(1, d/0.4) \times (1 - \mu)\). During the flash shock a central area buys at 50% off and anchors on $25.

🎯 The Manager's Question: Out of every 100 times the ad was shown, how many led to a click?

📊 Worked Business Example: At 0.5%, Plan A's 2,000,000 impressions bring 10,000 clicks. Raise Plan B to 1% at the same $5 CPM and it brings 20,000 clicks and 400 purchases.

⚖️ How to Judge the Number: 0% means nobody clicked. The tool starts both plans at 0.5%, so 1% doubles the clicks. Read it next to the CPM and the post-click purchase rate. In the "higher price" example Plan B doubles the click rate but pays a $12 CPM, and ends at $60 per purchase against $50.

⚠️ Common Misread to Avoid: Reading a higher click rate as more sales. Clicks can be curious or accidental, and a plan with double the click rate can still cost more per purchase if its CPM is higher.

📐 Technical Definition: The tool takes it as an assumed input in percent and computes expected clicks = impressions × CTR ÷ 100. It is not a benchmark; search ads usually run far higher than display.

🎯 The Manager's Question: How does the model put a category, such as region or channel, into an equation that needs numbers?

📊 Worked Business Example: In the regional store preset, Midwest is the baseline and West gets its own 0/1 column. Its coefficient of 13.9 means a West store sells about $13.9k more than a Midwest store with the same ad spend.

⚖️ How to Judge the Number: Each category except the baseline gets its own 0/1 column, and a coefficient of 0 means no gap from the baseline. The coefficient is that gap in units of {outcome}, with the other predictors held fixed. Read it with the category's row count: the West's 13.9 rests on only 6 stores.

⚠️ Common Misread to Avoid: Reading the coefficient as the category's own average. It is the gap from the baseline category, with the other predictors held fixed. West at +13.9 means 13.9 above a comparable Midwest store, not 13.9 in total.

📐 Technical Definition: A factor with m levels gives m minus 1 dummy columns in the design matrix, and the omitted level is the baseline. Including all m next to the intercept would make the columns add up to the intercept column, so the coefficients could not be solved. Dummy columns are left out of the variance inflation factor for the same reason.

🎯 The Manager's Question: Which pairs of groups really differ, once I allow for having compared so many?

📊 Worked Business Example: Four channels make six pairs. Paid social against Search has an unadjusted p of 0.59, which the table does not show; Holm raises it to 1.00, while all three pairs involving Referral stay below 0.001.

⚖️ How to Judge the Number: Read it against {alpha}, the same line as any other p-value; the adjustment is already applied, so do not correct it again. Read it next to the pairwise probability of superiority in the same row, which gives the size of each gap.

⚠️ Common Misread to Avoid: Reading an adjusted p of 1.00 as proof that Paid social and Search are identical. Holm multiplies small p-values and caps the result at 1, so 1.00 only means this data shows no clear gap between the two. Their medians, 10.9 and 9.3 days, still differ a little.

📐 Technical Definition: Each raw p comes from a z-test on the gap in mean ranks, \(z = (\bar{R}_a - \bar{R}_b) / SE\), with the overall tie-corrected standard error. Comparing six pairs at 0.05 each gives roughly a one-in-four chance of at least one false alarm when nothing differs, and Holm removes that inflation. It is sequential: the smallest raw p is multiplied by 6, the next by 5, and so on. The max in the formula keeps the adjusted values in the same order as the raw ones.

🎯 The Manager's Question: Which pairs of groups really differ, once I allow for having compared so many?

📊 Worked Business Example: Four channels make six pairs. Paid social against Search has an unadjusted p of 0.59, which the table does not show; Holm raises it to 1.00, while all three pairs involving Referral stay below 0.001.

⚖️ How to Judge the Number: Read it against {alpha}, the same line as any other p-value; the adjustment is already applied, so do not correct it again. Read it next to the pairwise probability of superiority in the same row, which gives the size of each gap.

⚠️ Common Misread to Avoid: Reading an adjusted p of 1.00 as proof that Paid social and Search are identical. Holm multiplies small p-values and caps the result at 1, so 1.00 only means this data shows no clear gap between the two. Their medians, 10.9 and 9.3 days, still differ a little.

📐 Technical Definition: Each raw p comes from a z-test on the gap in mean ranks, \(z = (\bar{R}_a - \bar{R}_b) / SE\), with the overall tie-corrected standard error. Comparing six pairs at 0.05 each gives roughly a one-in-four chance of at least one false alarm when nothing differs, and Holm removes that inflation. It is sequential: the smallest raw p is multiplied by 6, the next by 5, and so on. The max in the formula keeps the adjusted values in the same order as the raw ones.

🎯 The Manager's Question: Of the people who like one item, what share also like the other?

📊 Worked Business Example: Of Frost & Fold shoppers who would buy Mocha Almond, 86% would also buy Chocolate Fudge Brownie. Of Brownie fans, only 12% would also buy Raspberry Ripple, which is why that pair reaches so many people.

⚖️ How to Judge the Number: 1 means the second item adds no one new for the first item's fans, and 0 means none of them overlap. Read it next to each item's own acceptance rate: a high-overlap pair spends two slots on one audience.

⚠️ Common Misread to Avoid: Reading the table as symmetric. 81% of Chocolate Chip fans also like Brownie, but only 73% of Brownie fans like Chocolate Chip, because Brownie has more fans. Read each cell in its stated direction: of the row item’s fans, the share who also take the column item.

📐 Technical Definition: This is the conditional share \(P(B \mid A)\) over the acceptance matrix, counted without weights. The diagonal is 1 wherever an item has any acceptors at all.

🎯 The Manager's Question: In weekly data, does a miss one week tend to carry over into the next week?

📊 Worked Business Example: In the weekly ad-sales preset, 104 weeks give a Durbin-Watson of 0.58, far below 2. When sales beat the line one week, they usually beat it the next week too.

⚖️ How to Judge the Number: About 2 means no carry-over; toward 0 a miss tends to repeat next period, and toward 4 it tends to flip sign. Values below about 1.5 are a warning. Read it together with the lag-1 autocorrelation and the Ljung-Box test, which the tool uses for its verdict. Well below 2, the slope's standard errors are too small; add last week's sales or a time trend to the model.

⚠️ Common Misread to Avoid: Reading it on data with no time order. With ad spend picked as the time order, the 60 ad campaigns give 0.49, but that only echoes the curve. It means something only when rows run in real time order.

📐 Technical Definition: It is about \(2(1 - r_1)\), where \(r_1\) is the lag-1 autocorrelation of the residuals, so 0.58 goes with an \(r_1\) near 0.7. Exact p-values need tables that depend on the predictors, which is why the tool bases its verdict on Ljung-Box instead.

🎯 The Manager's Question: Is the network on the page the one from its best moment, or from the last pass?

📊 Worked Business Example: Tallyhop’s preset leaves early stopping off, and the network from epoch 300 scores a holdout log loss of 2.00. Turned on, it keeps epoch 9’s weights and scores 0.67, close to the regression’s 0.63.

⚖️ How to Judge the Number: On is the default, with a patience of 20 epochs: the page reports the best epoch’s network. Off is for watching overfitting happen, and every number then comes from the last epoch. Read the setting against the training chart: when the validation curve bottoms out and climbs, the gap between on and off is the cost of memorizing.

⚠️ Common Misread to Avoid: It does not wait for training loss to stop falling, which it rarely does. It watches validation loss, the curve that turns, and stops 20 epochs after the last improvement. At Ridgeline the best epoch is 22, so training ends at epoch 42.

📐 Technical Definition: After each epoch the tool keeps the weights with the lowest validation loss; an epoch counts as better only by beating the best by more than \(10^{-6}\). When the patience runs out it restores the kept weights. The generated Python code reproduces this with partial_fit on the same validation rows, not scikit-learn’s own early stopping.

🎯 The Manager's Question: How big is the difference, in a form I can compare across tests and studies?

📊 Worked Business Example: Aura's 3.8% against the 2.4% baseline is a Cohen's h of 0.08 at 1,850 clicks. Big Data Mode's 185,000 clicks push the p-value to almost zero, and h stays at 0.08.

⚖️ How to Judge the Number: 0 means no difference. Each family has its own scale; for Cohen's d and h the report panels call below 0.2 negligible, 0.2 small, 0.5 moderate and 0.8 large (Cohen, 1988). Read it next to the p-value, then translate it into money: Aura's h of 0.08 is negligible on that scale, yet 1.4 extra points on 200,000 clicks is worth about $74,000 a year.

⚠️ Common Misread to Avoid: Reading a tiny p-value as a big effect. Big Data Mode shrinks Aura's p-value from 0.00005 to almost zero while the lift stays 1.4 points and h stays 0.08. Sample size moves the p-value; only the effect size says how much.

📐 Technical Definition: A magnitude scaled so it does not grow with the sample size. Standardized versions also drop the units, so a click test and a survey can be compared. The one-sample tool reports Cohen's h for a proportion and Cohen's d for a mean. h rates a large relative lift near a low base rate as small (Aura's 58% lift is h 0.08), which is why the tool also states the lift in points and dollars. The bands are conventions, not laws, so a number is only small or large on a named scale.

🎯 The Manager's Question: Did the whole market end up somewhere that no single agent's rule asked for?

📊 Worked Business Example: In the default Brookhaven town every household wants only 33% same-group neighbors, yet the run settles at Year 21 with 71.7% same-group. The gap is +38.7 points of sorting that nobody asked for.

⚖️ How to Judge the Number: In the housing case the card shows segregation minus τ, so 0 means the town is exactly as sorted as its rules require; the default town starts at +0.7 points. The other cases show the change in the main outcome since Year 0, in percentage points. Read it next to the replication panel: a gap that holds across 30 seeds belongs to the rules, not to one lucky draw. A large positive gap warns that a survey of individual preferences will understate the market outcome.

⚠️ Common Misread to Avoid: Reading a large gap as a sign that agents want the extreme outcome. In Brookhaven nobody asks for more than a third of neighbors like them, yet 77% of households end with a same-group majority. The gap measures what the interactions add, not what anyone prefers.

📐 Technical Definition: In the business cases the card reads \(Y_T - Y_0\) instead, the change in the main outcome since Year 0. The gap is descriptive and carries no test of its own. One run is one draw, so use replications before calling a gap typical.

🎯 The Manager's Question: How many times has the network studied every training customer?

📊 Worked Business Example: Ridgeline has 2,399 training rows and a batch size of 64, so one epoch is 38 updates. Early stopping ends the run at epoch 42, about 1,600 updates in, and keeps epoch 22.

⚖️ How to Judge the Number: The maximum is 300 by default, and more is not better. Read the count next to the validation curve: the best epoch is where to stop, and with early stopping on the run ends 20 epochs after it.

⚠️ Common Misread to Avoid: More passes do not make a better model. After the best one, each extra pass memorizes a little more noise; at Tallyhop, validation loss more than triples between epoch 9 and epoch 300.

📐 Technical Definition: One full pass through the training rows in mini-batches. The rows are reshuffled before every epoch from their own seeded stream, so the same seed gives the same run.

🎯 The Manager's Question: How much of the ordering of {outcome} does the grouping account for?

📊 Worked Business Example: Across four channels, days to first order gives H = 62.5 from 240 sign-ups, so this effect size is 62.5 / 239 = 0.26. Most of it comes from Referral sign-ups ordering much sooner than the other three.

⚖️ How to Judge the Number: 0 means the groups are interleaved at random; 1 would mean the grouping sorts every case perfectly. Read it next to the pairwise table, which shows which groups drive it.

⚠️ Common Misread to Avoid: Reading 0.26 as "channel explains 26% of the days to first order". It is a share of the variation in the RANKS, not in the original units. It does not say the grouping explains that share of the days, the dollars or the rating points.

📐 Technical Definition: This is the form given by Tomczak and Tomczak (2014). It is graded on the same Cohen (1988) bands as eta squared, so the two can be read side by side.

🎯 The Manager's Question: Are our predictions equally precise for small accounts and large ones?

📊 Worked Business Example: In the B2B deal-size preset, deals from small clients miss by about $12k while deals from the largest clients miss by about $29k. The line still runs through the middle of both, but its standard errors assume one common spread.

⚖️ How to Judge the Number: The assumption holds when the spread of misses is the same at every fitted value and in every category. The tool treats a spread ratio of 2 or more between the widest and narrowest third as wide. Read it alongside the Breusch-Pagan p-value: both together mean the standard errors need fixing.

⚠️ Common Misread to Avoid: Thinking uneven spread makes the slope wrong. The slope stays unbiased; the standard errors, p-values and intervals are what go wrong. Robust standard errors or a log of the outcome are the usual repair.

📐 Technical Definition: It applies across categories too: each level of a factor should show the same spread. The tool checks it three ways: the Scale-Location plot, the Koenker Breusch-Pagan test, and the ratio of residual spreads across thirds of the fitted values and across levels. Its failure is called heteroscedasticity.

🎯 The Manager's Question: What share of the total variation in the outcome is accounted for by group membership?

📊 Worked Business Example: An η² of 0.14 means 14% of spend variation sits between the groups. The remaining 86% is within-group noise.

⚖️ How to Judge the Number: Compare against 0.01 (small), 0.06 (medium), and 0.14 (large) next to omega-squared (ω²) and the group means.

⚠️ Common Misread to Avoid: Reading η² as the gap between two specific groups. It summarizes all groups together and overestimates the population share in small samples.

📐 Technical Definition: In one-way ANOVA, \(\eta^2\) equals the \(R^2\) from regressing the outcome on group indicators. Sample \(\eta^2\) is biased upward in small samples.

🎯 The Manager's Question: What share of the total variation in the outcome is accounted for by group membership?

📊 Worked Business Example: An η² of 0.14 means 14% of spend variation sits between the groups. The remaining 86% is within-group noise.

⚖️ How to Judge the Number: Compare against 0.01 (small), 0.06 (medium), and 0.14 (large) next to omega-squared (ω²) and the group means.

⚠️ Common Misread to Avoid: Reading η² as the gap between two specific groups. It summarizes all groups together and overestimates the population share in small samples.

📐 Technical Definition: In one-way ANOVA, \(\eta^2\) equals the \(R^2\) from regressing the outcome on group indicators. Sample \(\eta^2\) is biased upward in small samples.

🎯 The Manager's Question: How did the gap between test and control markets change, period by period, around the launch?

📊 Worked Business Example: In Hearthstone Coffee the weeks before launch sit near 0 (minus $64 two weeks out). The gap then jumps to $2,571 in launch week and $3,413 the week after.

⚖️ How to Judge the Number: Each coefficient is the test-minus-control gap in that period, measured against the period just before launch, which is fixed at 0. Leads (before launch) should sit near 0: a lead as large as half the DiD estimate fails the pre-trend check, and Hearthstone's largest, minus $529, is 16% of $3,230. Lags show when the effect arrives and whether it lasts. Read them next to the pre-trend test, which checks all the leads at once.

⚠️ Common Misread to Avoid: Quoting one post-launch coefficient as "the effect". A single week is noisy: Hearthstone's lags run from $2,571 to $3,663, while the pooled DiD estimate is $3,230. Use the lags for timing and persistence and the DiD estimate for the size.

📐 Technical Definition: Standard errors are clustered by market, and intervals use \(G - 1\) degrees of freedom. The coefficients match statsmodels to 1e-4 on every preset. It assumes no anticipation: if customers react before launch, the leads absorb it.

🎯 The Manager's Question: Are very large misses more common than a bell curve would predict?

📊 Worked Business Example: For the 60 ad campaigns the excess kurtosis is 0.91, just under the tool's heavy-tail line of 1. The B2B deals-by-industry preset reaches 4.8, because Software deals miss far more widely than the rest.

⚖️ How to Judge the Number: Zero matches a bell curve. Above 1 means heavy tails, with more extreme misses than a bell curve, and the tool calls it severe above 3. Below zero means lighter tails. Read it next to the skewness and the Q-Q plot, which shows whether the tails or one stray case drive it. With heavy tails, check the extreme rows before trusting the p-values.

⚠️ Common Misread to Avoid: Reading it as how pointed the peak looks. It is driven by the tails: a few extreme misses can push it up on their own. In the deals-by-industry preset, the 4.8 comes from Software's wide misses, not from a sharp peak.

📐 Technical Definition: Here \(m_r = \frac{1}{n}\sum(e_i-\bar e)^r\), where \(e_i\) is the residual for case i and \(\bar e\) their mean. This matches scipy.stats.kurtosis with fisher=True and bias=False. Pooling groups with different spreads also produces heavy tails, so the tool asks you to fix unequal spread first.

🎯 The Manager's Question: How often does the second item turn up in a basket picked at random?

📊 Worked Business Example: Bananas sit in 3,407 of Corner Market's 5,000 baskets, so any rule ending in bananas has an expected confidence of 68.1%. A rule ending in pasta sauce has a far lower bar to clear, 17.5%.

⚖️ How to Judge the Number: It is the bar the Confidence column has to clear. Read it next to confidence: equal values mean the first item tells you nothing, and confidence divided by expected is the rule lift. When it is above 30% and the lift is below 1.2, the tool tags the rule "Popular item": confident only because the item is everywhere.

⚠️ Common Misread to Avoid: The Expected column is skipped as background. It is the other half of the comparison: 72.8% confidence against 68.1% expected is a lift of only 1.07.

📐 Technical Definition: It is \(P(Y)\) and does not depend on the starting item at all. The tool prints it beside confidence so the comparison is always in view.

🎯 The Manager's Question: How many customers would land in this category if the benchmark split were exactly right?

📊 Worked Business Example: With 1,250 taste testers and an equal split across 5 flavors, each flavor expects 1,250 times 0.20, or 250. A custom benchmark giving one flavor 30% would expect 375 there.

⚖️ How to Judge the Number: It is the yardstick each observed count is read against: O above E means the category beats the benchmark. Cochran's rule asks for every E to be at least 5 so the chi-square p-value can be trusted. Below that, merge small categories before testing.

⚠️ Common Misread to Avoid: Expecting E to be a whole number of people. It is a long-run average under the benchmark, so 1,250 testers over 3 flavors expects 416.7 each. Only the observed counts have to be whole.

📐 Technical Definition: The benchmark shares must add to 1, so the expected counts add to N, the same total as the observed counts. The tool rescales typed shares that do not add to 100%.

🎯 The Manager's Question: How many customers would land in this category if the benchmark split were exactly right?

📊 Worked Business Example: With 1,250 taste testers and an equal split across 5 flavors, each flavor expects 1,250 times 0.20, or 250. A custom benchmark giving one flavor 30% would expect 375 there.

⚖️ How to Judge the Number: It is the yardstick each observed count is read against: O above E means the category beats the benchmark. Cochran's rule asks for every E to be at least 5 so the chi-square p-value can be trusted. Below that, merge small categories before testing.

⚠️ Common Misread to Avoid: Expecting E to be a whole number of people. It is a long-run average under the benchmark, so 1,250 testers over 3 flavors expects 416.7 each. Only the observed counts have to be whole.

📐 Technical Definition: The benchmark shares must add to 1, so the expected counts add to N, the same total as the observed counts. The tool rescales typed shares that do not add to 100%.

🎯 The Manager's Question: How does the model adjust when customers or accounts are observed over unequal windows?

📊 Worked Business Example: In Mosaic Cloud, a 12-month account naturally logs more tickets than a 2-month account. Adding active_months as an exposure offset fixes the coefficient on ln(active_months) at 1.00 so IRRs compare tickets per active month.

⚖️ How to Judge the Number: An offset has a fixed slope of 1.00 on the log scale rather than an estimated parameter. Read predictor IRRs as rate multipliers per unit of exposure alongside the baseline rate.

⚠️ Common Misread to Avoid: Entering observation time as an ordinary predictor instead of an exposure offset. Estimating a free slope on raw months mixes duration with behavioral intensity instead of normalizing counts into a true per-month rate.

📐 Technical Definition: Moving \(\ln(E_i)\) to the left side gives \(\ln(\mu_i / E_i) = X_i\beta\). Exponentiating both sides shows that expected count equals exposure times the per-unit rate, \(\mu_i = E_i \exp(X_i\beta)\).

🎯 The Manager's Question: If the model were fitted without this case, how surprising would its value of {outcome} be?

📊 Worked Business Example: The $6.6k campaign has an externally studentized residual of -3.34. Allowing for the 60 campaigns checked, its Bonferroni p-value is 0.088, so it is not flagged as an outlier at 0.05.

⚖️ How to Judge the Number: Read it like a t value: 0 is no surprise and beyond plus or minus 3 is unusual. The Bonferroni test multiplies the p-value by the number of rows, because checking every row gives chance many tries. An adjusted p-value below {alpha} flags the case; read it next to Cook's distance to see whether the case also moves the fit.

⚠️ Common Misread to Avoid: Treating "flagged" as "delete". Check the row for a data-entry error first. If the value is real, report the results with and without it.

📐 Technical Definition: When the model holds, \(t_i\) follows a t distribution on \(n-k-1\) degrees of freedom. The tool reports the largest one with the Bonferroni p-value \(\min(1,\ n \cdot 2P(T_{n-k-1} > |t_i|))\), the same rule as car::outlierTest in R.

🎯 The Manager's Question: Does the model balance catching real yes customers against wasting contacts on no customers?

📊 Worked Business Example: At the 0.70 cutoff in the influencer preset precision is 89.4% and sensitivity 41.9%, giving an F1 of 0.57. A plain average would say 0.66; the F1 is pulled toward the weaker of the two.

⚖️ How to Judge the Number: It runs from 0 to 1, and 1 needs both perfect precision and perfect sensitivity. Flagging every post scores 0.63 here, the no-model floor, against 0.67 for the model at the 0.50 cutoff. Read it alongside precision and sensitivity, because one number hides which of the two is weak.

⚠️ Common Misread to Avoid: Treating a higher F1 as a better business result. It weighs a missed customer and a wasted contact equally, and ignores the customers correctly left alone. When a missed churner costs $45 and a retention offer $15, choose the cutoff on cost, not on F1.

📐 Technical Definition: Equivalently \(F_1 = 2TP/(2TP + FP + FN)\), which shows that true negatives never enter. It cannot be computed when there are no actual and no predicted yeses. The tool reports it at the cutoff in use.

🎯 The Manager's Question: Do the predictors, taken together, predict {outcome} better than just using its average?

📊 Worked Business Example: For the 60 ad campaigns F is about 378 on 1 and 58 degrees of freedom, with p below 0.001. Spend clearly beats the average, which says nothing yet about whether a straight line is the right shape.

⚖️ How to Judge the Number: If the predictors explained nothing, F would sit near 1. A p-value below {alpha} says they beat the average together; it does not say which one matters. Read it next to the individual coefficient tests and the diagnostics.

⚠️ Common Misread to Avoid: Reading a significant F as "the model is good". It only rules out the idea that none of the predictors matter. The ad campaigns pass it easily and still fail the linearity check.

📐 Technical Definition: Here \(SS_{\text{res}} = \sum e_i^2\), where \(e_i\) is the residual for case i. F is compared with an F distribution on \(k-1\) and \(n-k\) degrees of freedom, and assumes normal errors with equal spread.

🎯 The Manager's Question: Do the predictors, taken together, predict {outcome} better than just using its average?

📊 Worked Business Example: For the 60 ad campaigns F is about 378 on 1 and 58 degrees of freedom, with p below 0.001. Spend clearly beats the average, which says nothing yet about whether a straight line is the right shape.

⚖️ How to Judge the Number: If the predictors explained nothing, F would sit near 1. A p-value below {alpha} says they beat the average together; it does not say which one matters. Read it next to the individual coefficient tests and the diagnostics.

⚠️ Common Misread to Avoid: Reading a significant F as "the model is good". It only rules out the idea that none of the predictors matter. The ad campaigns pass it easily and still fail the linearity check.

📐 Technical Definition: Here \(SS_{\text{res}} = \sum e_i^2\), where \(e_i\) is the residual for case i. F is compared with an F distribution on \(k-1\) and \(n-k\) degrees of freedom, and assumes normal errors with equal spread.

🎯 The Manager's Question: How strongly does each item correlate with each factor, overlap included?

📊 Worked Business Example: In PureGlow, "Truthful ingredient labeling" correlates 0.82 with Authenticity and 0.29 with Efficacy, though its pattern loading on Efficacy is only 0.06. The extra comes through the 0.28 correlation between the two factors.

⚖️ How to Judge the Number: Each cell is a plain correlation from -1 to 1. With uncorrelated factors (Varimax) it equals the pattern matrix. Read it next to the pattern matrix and the factor correlations: assign items with the pattern matrix, and name a factor from the items that correlate with it most.

⚠️ Common Misread to Avoid: Assigning each item to the factor with its largest structure loading. Overlap between factors lifts every structure loading, so an item can correlate above 0.60 with two factors at once, as PureGlow's cross-loaders do. Use the pattern matrix to assign items.

📐 Technical Definition: When \(\Phi = I\), uncorrelated factors as after Varimax, the structure and pattern matrices are the same. The tool shows the factor correlations in their own table beside the loadings, with a toggle between the two matrices.

🎯 The Manager's Question: How many distinct ideas do these items measure, beyond what random answers would produce?

📊 Worked Business Example: In the default survey the first eigenvalue is 2.85 against a random-data benchmark of 1.54, and the second is 1.05 against 1.34. Parallel analysis keeps one factor; Kaiser's eigenvalue-above-1 rule would keep two.

⚖️ How to Judge the Number: Keep a factor only while its eigenvalue beats the 95th percentile of eigenvalues from random data of the same size, 500 simulated datasets here. The nothing-there benchmark is random data, not 1. Read the count next to the scree plot and the loadings: a factor with fewer than 3 clear items is too thin to name.

⚠️ Common Misread to Avoid: Trusting Kaiser's rule and keeping every factor with an eigenvalue above 1. In the default data that keeps a second factor at 1.05, though random answers alone reach 1.34 in that spot. Parallel analysis drops it, and the lab uses its count unless you override it.

📐 Technical Definition: The tool simulates 500 datasets of independent normal noise with the same n and p, seeded so a class sees the same benchmark, and takes the 95th percentile at each position. Eigenvalues come from the full correlation matrix. It counts every position whose real eigenvalue clears its benchmark, which in practice equals stopping at the first that does not.

🎯 The Manager's Question: Of all the alerts the model sends, what share are false alarms?

📊 Worked Business Example: In the churn preset, 98 of the 116 flagged subscribers were never going to cancel, so the false discovery rate is about 84.5%. At $20 a credit, that is $1,960 a month wasted per 1,000 subscribers.

⚖️ How to Judge the Number: It is 1 minus precision, so 0 means every alert is real and 0.5 means half are false. Read it against the cost of acting on a false alarm: at 84.5%, the retention credits mostly reward loyal customers. Lower it by scoring a higher-risk group, which raises the base rate.

⚠️ Common Misread to Avoid: Mixing it up with the false-positive rate. The false-positive rate divides false alarms by everyone without the event: 98 of 980, or 10%. The false discovery rate divides them by everyone flagged: 98 of 116, or 84.5%.

📐 Technical Definition: This is the screening version for one model at one threshold. It is not the Benjamini-Hochberg false discovery rate that corrects many significance tests at once, which the glossary explains under the Benjamini-Hochberg q-value. The lab computes it as 1 minus the exact precision.

🎯 The Manager's Question: Of the customers who would never say yes, what share do we contact anyway?

📊 Worked Business Example: At the 0.50 cutoff the influencer model flags 84 of the 537 posts that made no sale, an FPR of 15.6%. At 0.30 it flags 258 of them, 48.0%.

⚖️ How to Judge the Number: Lower is better, and 0 means no customer who says no is ever flagged. Flagging at random gives an FPR equal to the share flagged, the diagonal of the ROC curve. Read it alongside sensitivity at the same cutoff: a stricter cutoff lowers both.

⚠️ Common Misread to Avoid: Reading 15.6% as the share of flagged customers who were wrong. That share divides by flagged customers, 84 of 361 or 23.3%, which is 1 minus precision. The FPR divides by all actual noes instead.

📐 Technical Definition: It equals 1 minus specificity and is the horizontal axis of the ROC curve. It cannot be computed without actual noes.

🎯 The Manager's Question: Could a pattern this strong be luck, given how many rules were tested at once?

📊 Worked Business Example: At the default settings Corner Market tests 16,991 rules, and 242 survive the correction. Beer and diapers, the famous folklore pair, has q = 0.74: 29 shared baskets is close to the 24 that chance would give.

⚖️ How to Judge the Number: Judge it against 0.05, as you would a p-value, but the promise differs. Accept every rule with q below 0.05 and about 5% of the accepted rules are expected to be false, not a 5% chance of any false rule at all. Read it next to the basket count and rule lift, because a tiny q on 8 baskets is a real but useless pattern.

⚠️ Common Misread to Avoid: A small q is read as a rule worth acting on. It only says the pattern is probably not luck. Truffle oil and burrata have q below 0.001 on 8 baskets and are still not worth moving a shelf for.

📐 Technical Definition: The raw p-value is a one-sided Fisher exact test that the pair turns up together more often than chance. The tool applies Benjamini-Hochberg across every rule evaluated, before any display filter, because correcting after filtering would defeat the purpose. It controls the false discovery rate, not the chance of any single false rule.

🎯 The Manager's Question: How reliably is a word over-represented in one group compared with another after smoothing rare counts?

📊 Worked Business Example: In Canyon Creek Bistro, the word "cold" has z = +3.84 in Harborplace versus Downtown. It exceeds the 1.96 cutoff for distinctive Harborplace vocabulary.

⚖️ How to Judge the Number: Values beyond +1.96 or -1.96 mark statistically distinctive words at the 0.05 level, read alongside total word frequency.

⚠️ Common Misread to Avoid: Picking words by raw log-odds without shrinkage. Unsmoothed ratios let a word used 2 times versus 0 times outrank a word used 40 times versus 5 times.

📐 Technical Definition: Monroe, Colaresi, and Quinn (2008) place an informative Dirichlet prior proportional to overall corpus frequencies so rare words shrink toward zero.

🎯 The Manager's Question: What value of {outcome} does the model expect for a case with exactly these predictor values?

📊 Worked Business Example: For a $50k campaign the line predicts 105.76 + 0.665 × 50, about $139.0k in revenue. The campaigns near $50k actually landed above that, because the true curve bulges in the middle.

⚖️ How to Judge the Number: It is in the units of {outcome}, and the gap between it and the actual value is that case's residual. Read it next to the residual standard error: if the misses are bell-shaped, about two thirds of cases land within 1 typical miss of their fitted value. Diagnostic plots put it along the bottom because it sums up all the predictors at once.

⚠️ Common Misread to Avoid: Treating it as a forecast for a new case. The model was fitted on this very case, so fitted values look more accurate than a forecast for new data would be. Holdout or cross-validation errors show the honest forecast accuracy.

📐 Technical Definition: With one predictor this is \(b_0 + b_1 x_i\), where \(b_0\) is the intercept and \(b_1\) the slope. Dummy columns add their gap from the baseline for the case's category. The fitted values are the points on the fitted line or surface.

🎯 The Manager's Question: What is the lowest offer the publisher will take before leaving the ad space empty?

📊 Worked Business Example: The top banner's floor is $2.50 CPM, so FinBank's $1.80 bid is ineligible. Set the floor multiplier to 2 and the floor becomes $5.00: every bid fails and the space goes unsold.

⚖️ How to Judge the Number: A floor of $0 accepts any bid, and each step up removes the lowest offers first. Read it against the bids: under first price a floor below the top bid changes neither the winner nor the price, because the winner pays its own bid. A floor above the top bid leaves the space unsold, so watch the share of spaces that still sell.

⚠️ Common Misread to Avoid: Reading the floor as the price the winner pays. Under the first-price rule the winner pays its own bid: with a $2.50 floor, LuxeDerma still pays $4.20.

📐 Technical Definition: In the teaching auction a bid is eligible when its CPM is at least the floor CPM. The floor multiplier scales all three spaces (top banner $2.50, article $2.00, sidebar $3.00). Real platforms also apply fees, policy and timing rules before the comparison.

🎯 The Manager's Question: Which answer, such as converted or churned, are the predicted chances about?

📊 Worked Business Example: In the influencer preset the focal outcome is Converted = 1, a post that drove at least one sale: 463 of 1,000 posts. Every probability, odds ratio and sensitivity on the page is about that answer, not about the 537 posts with no sale.

⚖️ How to Judge the Number: Pick the event your decision acts on. For a text outcome the tool picks the less common answer by default, and the drop-down lets you change it. Read it next to the base rate: at 46.3% focal a 0.50 cutoff flags plenty of rows, while at 5% focal it may flag almost none.

⚠️ Common Misread to Avoid: Reading 1 as the good answer. If churn is coded 1, a higher prediction means more risk of losing the customer, and a positive coefficient is bad news. Flipping the focal answer flips the sign of every coefficient and turns each odds ratio into its reciprocal.

📐 Technical Definition: The selected category is mapped to 1 and the other to 0 before fitting. Swapping them changes the signs of the coefficients, not the fit: the deviance, AUC and log loss stay the same.

🎯 The Manager's Question: Does each factor hold its own items tightly and stay distinct from the other factors?

📊 Worked Business Example: In NovaBrew, once the three low-communality items are dropped, Social Image has AVE 0.65 and CR 0.85. Its square root of AVE, 0.80, is well above its largest correlation with another factor, 0.35, so it passes Fornell-Larcker.

⚖️ How to Judge the Number: An AVE of 0.50 is the line: below it, a factor explains less of its items than error does. A CR of 0.70 is the reliability line, read like alpha. For discriminant validity, read the square root of AVE against the factor's correlations with the other factors; it should beat every one. A failure means two factors overlap too much to report as separate ideas.

⚠️ Common Misread to Avoid: Treating CR as a second copy of alpha. CR is built from the factor loadings, so strong items count for more, and here it is the same number as that factor's omega. When CR and alpha differ, the factor's items have unequal loadings.

📐 Technical Definition: \(\text{CR} = (\sum \lambda_i)^2 / [(\sum \lambda_i)^2 + \sum (1 - \lambda_i^2)]\) over the same items. Fornell-Larcker passes when \(\sqrt{\text{AVE}}\) exceeds every absolute factor correlation \(\lvert\phi_{jk}\rvert\). The loadings come from the exploratory solution, so these are screens, and the tool does not compute HTMT, the stricter ratio the chapter recommends. Fornell-Larcker is a comparison rule, not a statistic of its own; it is listed here because the tool's Fornell-Larcker button opens this entry.

🎯 The Manager's Question: How many times should one person see our ad before we stop paying to show it again?

📊 Worked Business Example: With the simulator's cap of one ad per brand per page, LuxeDerma wins the top banner and is shut out of the next two spaces. GlowRoute's $3.80 then takes the article space instead of losing to LuxeDerma's $4.20.

⚖️ How to Judge the Number: A cap of 1 per page is strict; campaign caps are usually set per day or week, for example 3 per person per week. There is no universal best number. Read it next to average frequency and reach: a tighter cap spreads the same impressions over more different people.

⚠️ Common Misread to Avoid: Reading a cap of 3 as "each person saw the ad 3 times". A cap is a ceiling: many people see the ad once or twice, so the average frequency sits below 3.

📐 Technical Definition: A rule that drops an opportunity once a recognized user or device has hit the limit in the time window. Enforcement depends on identifiers and modeling, so a person on several devices can exceed it. In the simulator the cap applies across the three spaces on one page view.

🎯 The Manager's Question: Did the model hold up on rows it never saw?

📊 Worked Business Example: A churn model with a training AUC of 0.80 and a holdout AUC of 0.74 has lost 0.06 of its 0.30 edge over a coin toss, a 20% gap. The panel calls that worth watching.

⚖️ How to Judge the Number: Some drop is normal. Under 10% worse is fine, 10 to 25% is worth watching, and more than that means overfitting. For a number, worse means RMSE rose; for a yes-or-no outcome it is the share of the AUC's edge over 0.5 that was lost. For a category outcome it is the share of the accuracy edge over the baseline. Read it against the holdout size: a yes-or-no drop must also exceed twice the standard error of the holdout AUC before it is called overfitting.

⚠️ Common Misread to Avoid: Reading a holdout score as good as or better than training as proof of a great model. A small holdout can be lucky, so try another split seed before trusting a surprising result in either direction.

📐 Technical Definition: This is the yes-or-no form. For a numeric outcome the gap is the percentage rise in RMSE, \((\text{RMSE}_{\text{holdout}} - \text{RMSE}_{\text{train}})/\text{RMSE}_{\text{train}}\). With several categories, or a tree, the accuracy edge over the majority-class baseline replaces the AUC edge over 0.5. The AUC standard error follows Hanley and McNeil (1982).

🎯 The Manager's Question: If a customer buys this period, how much likelier does that make them to buy next period?

📊 Worked Business Example: In the Thistledown cold-brew data, treating every shopper alike says last week's purchase adds 29.9 points to this week's chance. Comparing each shopper with their own usual rate (model C), it adds only 4.2 points; the other 25.7 points were loyalty.

⚖️ How to Judge the Number: 0 points means no habit: last period's purchase only shows which customers are the buying kind. Compare model C with model A; the gap between them is loyalty, not habit. Read it next to the trial value, which turns these points into extra purchases per 100 trials and so into a yes or no on a trial campaign.

⚠️ Common Misread to Avoid: Quoting the pooled number (model A) as the habit. Stacking all customers together lets the loyal ones' long buying streaks pose as habit. In Thistledown that turns 4.2 points of real habit into 29.9.

📐 Technical Definition: It reads as habit only if the model has captured why customers differ, which is the job of models B and C. For model C each probability is also averaged over the customer baselines \(u_i \sim N(0, \sigma^2)\), so all three models sit on the same population-average scale. Models A and B match statsmodels; model C matches an independent quadrature fit.

🎯 The Manager's Question: At what weekly spend does this channel reach half of the most it could ever deliver?

📊 Worked Business Example: Brightwell's TV reaches half its maximum at about $24,000 a week (the slider's K of $60,000 × (1 - 0.6)) and averages $13,600: still on the steep part. Search reaches half at about $2,000 a week but averages $6,000, three times past it.

⚖️ How to Judge the Number: At a steady weekly spend of K × (1 - λ) a channel delivers 50% of its maximum, and the next dollar earns a quarter of what the first did. Below that spend there is headroom; at two or three times it most of the effect is used up. Compare that figure, not K itself, with the channel's average weekly spend, then confirm with marginal ROI.

⚠️ Common Misread to Avoid: Comparing K straight with weekly spend. K is measured in carried-over spend, which builds up over the weeks, so at a steady weekly spend half saturation arrives at K × (1 - λ). For TV that is $60,000 × 0.4 = $24,000 a week, not $60,000.

📐 Technical Definition: The tool uses the Hill curve with shape 1, which bends from the first dollar and has no S-shaped threshold. K is in adstocked units to match the fit; at a steady weekly spend x the carried-over level is \(x/(1 - \lambda)\), so half saturation comes at \(x = K(1 - \lambda)\). Response past the highest observed week is extrapolation, drawn dashed on the tool's curves.

🎯 The Manager's Question: How unusual is this case's mix of predictor values compared with the rest of the data?

📊 Worked Business Example: In the influencer preset, one celebrity partnership reached 450,000 people while the other 50 reached at most 80,000. Its leverage is 0.89, more than ten times the 0.078 cut-off.

⚖️ How to Judge the Number: Leverage runs from 1/n to 1, and the average is the number of coefficients divided by the number of rows. Twice that average counts as high: 2 × 2 / 51 = 0.078 for the influencer preset. It depends only on the predictors, so read it next to the standardized residual to see whether the case also misses. Before acting, refit without the high-leverage case and compare.

⚠️ Common Misread to Avoid: Treating every high-leverage case as a problem. A far-out case that sits on the pattern is harmless and even steadies the slope. It becomes a problem when it also misses, which is what Cook's distance measures.

📐 Technical Definition: With one predictor it simplifies to \(1/n + (x_i - \bar{x})^2 / \sum_j (x_j - \bar{x})^2\), where n is the number of cases: leverage is distance from the average predictor value. Leverages sum to k, the number of coefficients, so the average is k/n and the tool flags values above 2k/n.

🎯 The Manager's Question: How unusual is this case's mix of predictor values compared with the rest of the data?

📊 Worked Business Example: In the influencer preset, one celebrity partnership reached 450,000 people while the other 50 reached at most 80,000. Its leverage is 0.89, more than ten times the 0.078 cut-off.

⚖️ How to Judge the Number: Leverage runs from 1/n to 1, and the average is the number of coefficients divided by the number of rows. Twice that average counts as high: 2 × 2 / 51 = 0.078 for the influencer preset. It depends only on the predictors, so read it next to the standardized residual to see whether the case also misses. Before acting, refit without the high-leverage case and compare.

⚠️ Common Misread to Avoid: Treating every high-leverage case as a problem. A far-out case that sits on the pattern is harmless and even steadies the slope. It becomes a problem when it also misses, which is what Cook's distance measures.

📐 Technical Definition: With one predictor it simplifies to \(1/n + (x_i - \bar{x})^2 / \sum_j (x_j - \bar{x})^2\), where n is the number of cases: leverage is distance from the average predictor value. Leverages sum to k, the number of coefficients, so the average is k/n and the tool flags values above 2k/n.

🎯 The Manager's Question: Of the customers who made it to this {unit}, what share left during it?

📊 Worked Business Example: In Parcel & Pantry, month one loses 338 of 1,587 customers, a hazard of 21%. From month two on it hovers near 6%, with a median of 5.8% across the later months.

⚖️ How to Judge the Number: A flat line means risk does not change with tenure. The tool flags a cliff when the worst period is at least 2 times the median of the others; here month one is about 3.7 times. Read it next to the survival curve: a cliff puts the budget in the first weeks, not in perks for long-standing customers.

⚠️ Common Misread to Avoid: Reading a hazard as the share of all customers who left. Its denominator is only those still present, so the 7.5% hazard at month 22 here is 7 people out of 93. A high late hazard can mean very few losses.

📐 Technical Definition: This is the discrete-time hazard. Survival is the running product of \(1 - h(t)\), so one bad early period lowers the whole curve. The cliff check takes the worst period among those with at least 20 customers still at risk. It compares that period with the median of the rest, so one noisy month can neither hide a spike nor, at the thin end of the curve, fake one.

🎯 The Manager's Question: Would asking every buyer at once get the publisher a better price than asking one by one?

📊 Worked Business Example: Asked together, the five top-banner offers compete and LuxeDerma's $4.20 CPM wins. Asked in order, GlowRoute comes first and its $3.80 is accepted before LuxeDerma is ever asked.

⚖️ How to Judge the Number: Nothing is gained when the first buyer asked already has the top bid; then both methods give the same price. Here asking together earns $4.20 instead of $3.80 per 1,000, about 10% more. Read that gain against page-load time and fees, which the simulator does not model.

⚠️ Common Misread to Avoid: Reading the 10% gain in this example as what header bidding adds on any site. It depends on how often a later buyer outbids the first one, and real setups add timeouts, fees and several auction layers.

📐 Technical Definition: A wrapper, in the page or on a server, collects bids from several selling platforms. It passes them to the publisher's ad server for the final choice. The simulator's "together" option compares the same offers in one step and leaves out those layers and the timing.

🎯 The Manager's Question: How many standard deviations big is the effect after correcting for small-sample exaggeration?

📊 Worked Business Example: In a 20-pair test where Cohen’s d_z is 1.69, the sample SD slightly understates true spread. Hedges’ g trims that estimate to 1.62.

⚖️ How to Judge the Number: Judge it on the 0.20 / 0.50 / 0.80 scale next to the raw mean difference and confidence interval.

⚠️ Common Misread to Avoid: Treating Hedges’ g and Cohen’s d as testing different hypotheses. They measure the same gap and only diverge when samples are under 30.

📐 Technical Definition: Sample standard deviations underestimate population spread in small samples, inflating Cohen’s d. Multiplying by the correction factor removes almost all of that small-sample bias.

🎯 The Manager's Question: How many standard deviations big is the effect after correcting for small-sample exaggeration?

📊 Worked Business Example: In a 20-pair test where Cohen’s d_z is 1.69, the sample SD slightly understates true spread. Hedges’ g trims that estimate to 1.62.

⚖️ How to Judge the Number: Judge it on the 0.20 / 0.50 / 0.80 scale next to the raw mean difference and confidence interval.

⚠️ Common Misread to Avoid: Treating Hedges’ g and Cohen’s d as testing different hypotheses. They measure the same gap and only diverge when samples are under 30.

📐 Technical Definition: Sample standard deviations underestimate population spread in small samples, inflating Cohen’s d. Multiplying by the correction factor removes almost all of that small-sample bias.

🎯 The Manager's Question: Are our predictions equally precise for small accounts and large ones?

📊 Worked Business Example: In the B2B deal-size preset, deals from small clients miss by about $12k while deals from the largest clients miss by about $29k. The line still runs through the middle of both, but its standard errors assume one common spread.

⚖️ How to Judge the Number: The assumption holds when the spread of misses is the same at every fitted value and in every category. The tool treats a spread ratio of 2 or more between the widest and narrowest third as wide. Read it alongside the Breusch-Pagan p-value: both together mean the standard errors need fixing.

⚠️ Common Misread to Avoid: Thinking uneven spread makes the slope wrong. The slope stays unbiased; the standard errors, p-values and intervals are what go wrong. Robust standard errors or a log of the outcome are the usual repair.

📐 Technical Definition: It applies across categories too: each level of a factor should show the same spread. The tool checks it three ways: the Scale-Location plot, the Koenker Breusch-Pagan test, and the ratio of residual spreads across thirds of the fitted values and across levels. Its failure is called heteroscedasticity.

🎯 The Manager's Question: How much room does the network have to bend its predictions?

📊 Worked Business Example: Ridgeline’s default, one layer of 16 units on 13 inputs, has 241 parameters for 2,399 training rows. Tallyhop’s preset of two layers of 32 units on 21 inputs has 1,793 for just 239 training trials.

⚖️ How to Judge the Number: The default is 1 layer of 16 units; the tool allows 2 to 64 units and up to 2 layers. More units bend more and memorize more. Read it next to parameters per training row: above 1, the network can memorize every row.

⚠️ Common Misread to Avoid: More units do not make a smarter network. On a small table they mostly add room to memorize: Tallyhop’s 2 × 32 network ends with training loss 0.006 and holdout log loss 2.00.

📐 Technical Definition: Each unit computes a weighted sum of the previous layer plus a bias and passes it through the activation. So each unit adds one weight per incoming value plus one bias. Ridgeline’s hidden layer adds \((13 + 1) \times 16 = 224\) and the output unit 17 more, 241 in all.

🎯 The Manager's Question: How far off is the model, in percent, on recent weeks it never saw while fitting?

📊 Worked Business Example: The default Brightwell model trains on 91 weeks and holds out the last 13. It misses those 13 weeks by 0.9% on average, against 0.5% on the training weeks.

⚖️ How to Judge the Number: 0% is a perfect forecast. The forecasting chapter calls under 5% excellent and 5% to 15% good enough for budgeting. Read it against the training MAPE: the tool warns when holdout error is more than twice the training error, a sign the model learned noise.

⚠️ Common Misread to Avoid: Taking a low holdout MAPE as evidence that the channel credits are right. A model can forecast total sales well while splitting credit wrongly between channels whose spend moves together. Accuracy checks the forecast; only experiments such as geo holdouts check the credit.

📐 Technical Definition: Adstock runs over the full series first, so the holdout weeks inherit carryover from training weeks, while the coefficients are fitted on training weeks only. MAPE breaks down when actual sales are near 0. The holdout length can be changed in the model settings.

🎯 The Manager's Question: How much better than guessing the average does the model do on rows it never saw?

📊 Worked Business Example: In the EDM event-spend preset the training R² is 0.56. One random 20% holdout scored 0.61, higher than training, which is the luck of a small split rather than a better model.

⚖️ How to Judge the Number: 1 is perfect and 0 means no better than guessing the held-back rows' own average. It can go negative, which means the model does worse than that guess. Read it against the training R²: expect it a little lower, and treat a big drop as overfitting.

⚠️ Common Misread to Avoid: Reading it with the usual 0-to-1 intuition. A negative value is not a bug; it is a model that fails on new data. And a holdout figure above training, like 0.61 against 0.56, is usually a lucky split.

📐 Technical Definition: Both sums use the held-back rows only, and the result is not clamped at 0. With a transformed outcome both are in real units, so neither will match the R² in the model summary, which is on the transformed scale.

🎯 The Manager's Question: Does this pair still stand out once we account for testing many pairs at once?

📊 Worked Business Example: In a 5-variable matrix there are 10 pairs, so a 5% alpha gives a 40% chance of at least one false alarm. Holm multiplies the smallest p-value by 10, the next by 9, and so on.

⚖️ How to Judge the Number: Compare the adjusted p-value against {alpha}, just like a single-test p-value. Read it next to the raw p-value to see which marginal pairs were likely screening noise.

⚠️ Common Misread to Avoid: Treating a pair that loses significance after Holm correction as proven unrelated. The correction keeps the family-wise false-alarm rate at {alpha}, which makes weak signals harder to confirm.

📐 Technical Definition: Holm's step-down rule controls the family-wise error rate under any dependence structure. It is always at least as powerful as the classic Bonferroni cutoff \(\alpha / m\).

🎯 The Manager's Question: Does this pair still stand out once we account for testing many pairs at once?

📊 Worked Business Example: In a 5-variable matrix there are 10 pairs, so a 5% alpha gives a 40% chance of at least one false alarm. Holm multiplies the smallest p-value by 10, the next by 9, and so on.

⚖️ How to Judge the Number: Compare the adjusted p-value against {alpha}, just like a single-test p-value. Read it next to the raw p-value to see which marginal pairs were likely screening noise.

⚠️ Common Misread to Avoid: Treating a pair that loses significance after Holm correction as proven unrelated. The correction keeps the family-wise false-alarm rate at {alpha}, which makes weak signals harder to confirm.

📐 Technical Definition: Holm's step-down rule controls the family-wise error rate under any dependence structure. It is always at least as powerful as the classic Bonferroni cutoff \(\alpha / m\).

🎯 The Manager's Question: Are our predictions equally precise for small accounts and large ones?

📊 Worked Business Example: In the B2B deal-size preset, deals from small clients miss by about $12k while deals from the largest clients miss by about $29k. The line still runs through the middle of both, but its standard errors assume one common spread.

⚖️ How to Judge the Number: The assumption holds when the spread of misses is the same at every fitted value and in every category. The tool treats a spread ratio of 2 or more between the widest and narrowest third as wide. Read it alongside the Breusch-Pagan p-value: both together mean the standard errors need fixing.

⚠️ Common Misread to Avoid: Thinking uneven spread makes the slope wrong. The slope stays unbiased; the standard errors, p-values and intervals are what go wrong. Robust standard errors or a log of the outcome are the usual repair.

📐 Technical Definition: It applies across categories too: each level of a factor should show the same spread. The tool checks it three ways: the Scale-Location plot, the Koenker Breusch-Pagan test, and the ratio of residual spreads across thirds of the fitted values and across levels. Its failure is called heteroscedasticity.

🎯 The Manager's Question: Group by group, do the rates that actually happened match the chances the model predicted?

📊 Worked Business Example: In the influencer-campaign scenario, 1,000 posts fall into 10 groups of 100 by predicted chance; the top group expected 97.5 conversions and got 95. Over all groups the statistic is 9.24 on 8 degrees of freedom, p = 0.32: no clear mismatch.

⚖️ How to Judge the Number: A p-value below {alpha} says the predicted chances miss the observed rates somewhere; a larger one says this grouped test found no clear mismatch. Read it next to the actual-against-fitted calibration chart, which shows where any misses are. The tool warns when a group expects fewer than 5 events or non-events, where the test is shaky.

⚠️ Common Misread to Avoid: A large p-value does not show the probabilities are right. Grouping choices move the result, small samples hide problems, and with tens of thousands of rows it flags mismatches too small to matter.

📐 Technical Definition: Rows are sorted by predicted probability and cut into equal-size groups, fewer than 10 for small samples. The statistic is compared with a chi-square on \(G - 2\) degrees of freedom, counting only the groups actually used.

🎯 The Manager's Question: Do different marketing levers drive whether a customer acts at all versus how often they repeat?

📊 Worked Business Example: In Copperpaw Pets, a first-order coupon lifts Stage 1 retention odds (OR = 1.88) but lowers Stage 2 repeat bag rate among active buyers (IRR = 0.78). A single-stage model blurs those two opposing effects together.

⚖️ How to Judge the Number: Compare Stage 1 odds ratios against 1.00 and Stage 2 IRRs against 1.00 alongside the single-stage model AIC. When a predictor flips direction across stages, acquisition and repeat behavior follow separate processes.

⚠️ Common Misread to Avoid: Reading the Stage 2 zero-truncated IRR as the effect across all customers including non-buyers. Stage 2 conditions strictly on active customers with Y >= 1, while overall expected count multiplies Stage 1 conversion by Stage 2 intensity.

📐 Technical Definition: The joint log-likelihood separates cleanly into a binary logit piece for \(Y_i > 0\) and a zero-truncated Poisson piece for positive counts. Both stages are estimated by maximum likelihood, and their log-likelihoods sum to give total model fit.

🎯 The Manager's Question: If this driver is switched on today, how many points does today's chance of a "yes" move?

📊 Worked Business Example: At Kestrel a truck day cuts the chance of an empty shelf by 17 points on average. After a day that was already empty the cut is 42 points; after a stocked day it is only 12.

⚖️ How to Judge the Number: 0 means the driver does not move today's chance. The size is in percentage points, so 17 points on an 18% base rate is large. Read the two splits against each other: when the effect after a "yes" is several times the effect after a "no", the driver matters most in that state.

⚠️ Common Misread to Avoid: Reading the 17-point effect as what a delivery is worth over time. It is the effect on the same day only. A stockout avoided today also lowers tomorrow's chance, so the lasting effect builds on it, and the long-run rate is the number for a recurring plan.

📐 Technical Definition: Everything else stays at its observed value in each period, including last period's outcome. For a numeric driver the step is one standard deviation instead of 0 to 1. Split by \(y_{t-1}\), the same average is taken separately over the periods that followed a "no" and a "yes".

🎯 The Manager's Question: By what factor does the expected event count multiply for a one-unit step in this predictor?

📊 Worked Business Example: In CraveDrop, the DashPass membership slope is 0.489, giving an IRR of exp(0.489) = 1.63. Holding other predictors fixed, members place 1.63 times as many quarterly orders (+63%) as non-members.

⚖️ How to Judge the Number: 1.00 means no change in expected count. Above 1.00 multiplies the expected rate upward and below 1.00 cuts it, read alongside the confidence interval and baseline count.

⚠️ Common Misread to Avoid: Reading an IRR of 1.63 as 1.63 additional orders for every customer. Because an IRR is a multiplier, it adds 1.26 orders to a 2-order baseline and 5.04 orders to an 8-order baseline.

📐 Technical Definition: Because the model uses a natural log link, adding \(\hat{\beta}_j\) on the log scale multiplies the expected count \(\hat{\mu}\) by \(e^{\hat{\beta}_j}\). Percentage change equals \((e^{\hat{\beta}_j} - 1) \times 100\%\).

🎯 The Manager's Question: What does each item add on top of the ones already chosen?

📊 Worked Business Example: Frost & Fold's ladder starts with Chocolate Fudge Brownie at 48.4%. Raspberry Ripple then adds 27.8 points and Strawberries & Cream adds 9.1, for 85.3% in total.

⚖️ How to Judge the Number: Steps that shrink are the normal pattern of diminishing returns. A step near 0 means the item reaches almost nobody the earlier ones missed. Read it next to the reach curve, which shows where one more item stops paying for its slot.

⚠️ Common Misread to Avoid: Reading the ladder order as a ranking of importance. The ladder adds the biggest gain first, so an item placed third may be strong on its own and simply overlap the first two. The steps only make sense in this order.

📐 Technical Definition: The order is greedy within the chosen set: at each step the item adding the most reach goes next. The steps sum to the reach of the whole set, and a different order gives different steps with the same total.

🎯 The Manager's Question: Does the route through the mediator grow or shrink as the moderator changes, and by how much?

📊 Worked Business Example: In the influencer case, trust carries 0.35 of the effect for 23-year-olds but 0.18 for 45-year-olds. The index is 0.48 × -0.18 = -0.09: the route shrinks by about 0.09 for each standard deviation of age.

⚖️ How to Judge the Number: Zero means the route is the same size at every level of the moderator, and its interval against 0 is the formal test. Read it next to the conditional indirect effects at low, average and high values of the moderator, which show where the route is worth acting on. A clearly negative index for age says to aim the influencer budget at younger buyers.

⚠️ Common Misread to Avoid: A route that is significant for younger buyers and not for older ones is read as moderated mediation. That comparison is not a test, because a difference in significance is not a significant difference. Only the index, or the interval of the gap between two conditional effects, answers it.

📐 Technical Definition: This is the b-path version (PROCESS model 14); when only the a path is moderated (model 7) the index is \(a_3 b\). Both assume the interaction is linear in the moderator. With both paths moderated the route is quadratic in the moderator and no single index sums it up. This tool plots the conditional indirect effect across the moderator rather than printing the index.

🎯 The Manager's Question: How does the model put a category, such as region or channel, into an equation that needs numbers?

📊 Worked Business Example: In the regional store preset, Midwest is the baseline and West gets its own 0/1 column. Its coefficient of 13.9 means a West store sells about $13.9k more than a Midwest store with the same ad spend.

⚖️ How to Judge the Number: Each category except the baseline gets its own 0/1 column, and a coefficient of 0 means no gap from the baseline. The coefficient is that gap in units of {outcome}, with the other predictors held fixed. Read it with the category's row count: the West's 13.9 rests on only 6 stores.

⚠️ Common Misread to Avoid: Reading the coefficient as the category's own average. It is the gap from the baseline category, with the other predictors held fixed. West at +13.9 means 13.9 above a comparable Midwest store, not 13.9 in total.

📐 Technical Definition: A factor with m levels gives m minus 1 dummy columns in the design matrix, and the omitted level is the baseline. Including all m next to the intercept would make the columns add up to the intercept column, so the coefficients could not be solved. Dummy columns are left out of the variance inflation factor for the same reason.

🎯 The Manager's Question: How much of what {predictor} does to {outcome} travels through the mediator?

📊 Worked Business Example: In the Artisan Bean case, price lifts perceived quality (a = 0.44) and quality lifts willingness to pay (b = 0.58). The indirect effect is 0.44 × 0.58 = 0.26 standard deviations of willingness to pay per standard deviation of price.

⚖️ How to Judge the Number: Zero means nothing travels through the mediator. The test is whether its interval excludes zero, not whether the a and b paths are each significant on their own. For size, read it against the total effect: an indirect effect of 0.26 inside a total of 0.57 means 46% of the effect flows through the mediator. That share tells you whether working on the mediator, such as quality cues, is worth the budget.

⚠️ Common Misread to Avoid: A clear indirect effect is read as proof that the mediator is the mechanism. It is a pattern of association. The same 0.26 would appear if willingness to pay drove the quality ratings, or if something unmeasured drove both, so the mechanism claim needs time order or an experiment.

📐 Technical Definition: Assumes both equations are correctly specified, with nothing unmeasured driving both the mediator and {outcome}. The tool fits \(M = i_M + aX\) and \(Y = i_Y + c'X + bM\) by ordinary least squares, the same model as PROCESS model 4. A product of two estimates has a skewed sampling distribution, so the tool reports a Monte Carlo interval from typed estimates or a percentile bootstrap from raw rows. The symmetric delta-method interval is shown only for comparison.

🎯 The Manager's Question: Why does the model use whether each customer bought in the first period we observe?

📊 Worked Business Example: In Thistledown, a shopper who bought in week 1 has about 5.8 times the odds of buying in any later week, other terms equal (model C coefficient 1.76). That is week 1 flagging the regulars, not week 1 changing anyone.

⚖️ How to Judge the Number: A coefficient of 0 would mean the first period says nothing about the customer's baseline. A large positive value, like Thistledown's 1.76, means first-period buyers are the buying kind. It is a required control, not a finding: read the habit row next to it, never instead of it.

⚠️ Common Misread to Avoid: Reading its coefficient as the effect of a first purchase, as if week 1 were special. It stands in for the part of each customer's baseline that the first period reveals. Leaving it out lets that baseline leak into the habit estimate.

📐 Technical Definition: Wooldridge (2005): modelling the baseline given \(y_{i0}\) and \(\bar{x}_i\) handles the initial-conditions problem of dynamic panel models. The problem is that the first period you see is already shaped by who the customer is. That is why each customer's first period is used only as history and the fit starts at their second.

🎯 The Manager's Question: What value of {outcome} does the model predict with every numeric predictor at zero and every category at baseline?

📊 Worked Business Example: For the ad campaigns the intercept is $105.8k, the revenue the line predicts for a campaign with no spend. No campaign spent under $6.6k, so treat it as an anchor for the line, not a forecast.

⚖️ How to Judge the Number: It means something only when 0 on every numeric predictor lies inside the data. Read it next to the smallest predictor values: if no case is near zero, it is an extrapolation that just anchors the line. In a model with categories, it is the prediction for the baseline category.

⚠️ Common Misread to Avoid: Reading it as the starting or lowest value of {outcome}. It is often a point no case comes near, so an odd value, even a negative one, is not a sign the model is wrong. Centring the predictor moves it to the average case without changing the fit.

📐 Technical Definition: That is the one-predictor form. With several predictors it is the first element of \(\mathbf{b}\), where \(\mathbf{b}\) is the vector of estimated coefficients. It equals the fitted value for a case at 0 on every numeric predictor and in the baseline level of every factor.

🎯 The Manager's Question: Measured in typical misses, how unusual is this case's miss?

📊 Worked Business Example: The $6.6k campaign missed by $21.6k against a typical miss of $7.26k, a standardized residual of -3.08. It is one of 3 campaigns past plus or minus 2, where about 3 of 60 are expected by chance.

⚖️ How to Judge the Number: Zero is a perfect hit. When the model is right, about 95% of cases sit within plus or minus 2, and beyond plus or minus 3 deserves a look at that row. Read the count past 2 against the number expected, about 4.6% of the rows.

⚠️ Common Misread to Avoid: Treating one value past 3 as proof the misses are not normal. In a sample of a few hundred, one such case often turns up by chance. Read the whole Q-Q plot, not one point.

📐 Technical Definition: High-leverage cases pull the line toward themselves, so their raw residuals run small. Dividing by \(\sqrt{1-h_{ii}}\) puts every case on one scale. In the influencer preset the celebrity's raw miss is 2.2 typical misses, but its standardized residual is -6.73.

🎯 The Manager's Question: When we know what really happened, did customers land inside their ranges as often as promised?

📊 Worked Business Example: Score 40 held-back customers and 37 land inside their own 95% ranges, 92.5% coverage. With 40 rows anything from 36 to 40 inside is ordinary chance, so the ranges pass.

⚖️ How to Judge the Number: About 95% is honest. A small sample wanders by chance: with 40 rows, anything from 36 to 40 inside (90% to 100%) is unremarkable, so the verdict allows for that before calling the ranges wrong. Misses should split roughly evenly above and below; if nearly all fall on one side, the spread is lopsided, and a log of the outcome often fixes it. Read it against held-back rows or a new file, because coverage on the training rows flatters the model: those rows helped set the width.

⚠️ Common Misread to Avoid: Higher is not better. On a large holdout (about 80 rows or more), 100% means the ranges are wider than they need to be; on a small one it is ordinary luck.

📐 Technical Definition: It is compared with the nominal 95% allowing two binomial standard errors, \(2\sqrt{0.95 \times 0.05 / n}\). A one-sided split of the misses is checked with an exact two-sided binomial test against 50/50.

🎯 The Manager's Question: How spread out are the middle half of my cases?

📊 Worked Business Example: Flow A's middle half of orders runs from $43.94 to $92.09, an IQR of $48.15, and Flow B's from $60.13 to $127.83, an IQR of $67.70. The $4,120 bulk order in Flow A barely moves its IQR.

⚖️ How to Judge the Number: 0 means the middle half of cases share one value; larger means more variable. Read it next to the median it surrounds and against the other group: Flow B's IQR is 1.4 times Flow A's. Unlike the standard deviation, one extreme value cannot inflate it.

⚠️ Common Misread to Avoid: Reading every value outside the IQR as unusual. Half the cases fall outside it by construction: 48 of Flow A's 95 orders. Unusual starts at the Tukey fences, 1.5 IQRs beyond the quartiles, which puts Flow A's upper fence near $164.

📐 Technical Definition: The Tukey fences at \(Q_1 - 1.5 \times IQR\) and \(Q_3 + 1.5 \times IQR\) flag points worth a look; they are a convention, not a test. The tool's quartiles interpolate between sorted values, as Excel's QUARTILE.INC does, so other software can differ by a few cents.

🎯 The Manager's Question: Did this respondent read each matrix item, or click the exact same column straight down?

📊 Worked Business Example: In Bramblecup Cold Brew, a respondent who clicks 5, 5, 5, 5, 5 across a five-item brand matrix has IRV = 0.00. A thoughtful reader who answers 5, 4, 5, 4, 2 has IRV = 1.22.

⚖️ How to Judge the Number: 0.00 indicates a complete flatline across the block. Interpret 0.00 alongside block length and reverse-worded item presence before scrubbing.

⚠️ Common Misread to Avoid: Deleting every respondent with IRV = 0.00 on a short four-item unidirectional scale. Loyal brand advocates often rate all four positive items 5 out of 5 genuinely, whereas flatlining across eight items or a reverse-worded item signals true carelessness.

📐 Technical Definition: IRV computes the sample standard deviation of a single respondent's raw answers across a multi-item battery before reverse-coding. Because random flatline probability scales as \(K^{-(m-1)}\), diagnostic severity increases sharply with block length \(m\).

🎯 The Manager's Question: Does this item rise and fall with the rest of the scale?

📊 Worked Business Example: In the default survey, "Support green policies" correlates 0.53 with the total of the other seven items. "Watch sports on weekend" correlates 0.02: people who care about the environment are no more or less likely to watch sports.

⚖️ How to Judge the Number: 0 means the item is unrelated to the rest of the scale. The tool flags anything below 0.30 as weak; the chapter's stricter screen is 0.40. A negative value usually means the item is worded the other way round. Read it next to alpha if item deleted: a weak item normally raises alpha when it goes.

⚠️ Common Misread to Avoid: Deleting an item because its correlation is negative. Negative usually means it is worded in reverse, such as "The setup was confusing" among positive items. Tick it for reverse coding first; it then often correlates as well as the rest.

📐 Technical Definition: "Corrected" means the item is left out of the total it is correlated with; leaving it in would inflate the correlation, most of all on short scales. It is a Pearson correlation on complete responses and matches the Python reference to six decimals.

🎯 The Manager's Question: Does this item rise and fall with the rest of the scale?

📊 Worked Business Example: In the default survey, "Support green policies" correlates 0.53 with the total of the other seven items. "Watch sports on weekend" correlates 0.02: people who care about the environment are no more or less likely to watch sports.

⚖️ How to Judge the Number: 0 means the item is unrelated to the rest of the scale. The tool flags anything below 0.30 as weak; the chapter's stricter screen is 0.40. A negative value usually means the item is worded the other way round. Read it next to alpha if item deleted: a weak item normally raises alpha when it goes.

⚠️ Common Misread to Avoid: Deleting an item because its correlation is negative. Negative usually means it is worded in reverse, such as "The setup was confusing" among positive items. Tick it for reverse coding first; it then often correlates as well as the rest.

📐 Technical Definition: "Corrected" means the item is left out of the total it is correlated with; leaving it in would inflate the correlation, most of all on short scales. It is a Pearson correlation on complete responses and matches the Python reference to six decimals.

🎯 The Manager's Question: How many distinct ideas do these items measure, beyond what random answers would produce?

📊 Worked Business Example: In the default survey the first eigenvalue is 2.85 against a random-data benchmark of 1.54, and the second is 1.05 against 1.34. Parallel analysis keeps one factor; Kaiser's eigenvalue-above-1 rule would keep two.

⚖️ How to Judge the Number: Keep a factor only while its eigenvalue beats the 95th percentile of eigenvalues from random data of the same size, 500 simulated datasets here. The nothing-there benchmark is random data, not 1. Read the count next to the scree plot and the loadings: a factor with fewer than 3 clear items is too thin to name.

⚠️ Common Misread to Avoid: Trusting Kaiser's rule and keeping every factor with an eigenvalue above 1. In the default data that keeps a second factor at 1.05, though random answers alone reach 1.34 in that spot. Parallel analysis drops it, and the lab uses its count unless you override it.

📐 Technical Definition: The tool simulates 500 datasets of independent normal noise with the same n and p, seeded so a class sees the same benchmark, and takes the 95th percentile at each position. Eigenvalues come from the full correlation matrix. It counts every position whose real eigenvalue clears its benchmark, which in practice equals stopping at the first that does not.

🎯 The Manager's Question: What share of customers is still with us at each age since they signed up?

📊 Worked Business Example: In Parcel & Pantry the curve drops to 79% after month one, then loses about 4 points a month through the first year. It crosses 50% at month 8 and sits at 28% by month 18.

⚖️ How to Judge the Number: Start at 100% and read the shape. A steep first step and a gentler slope after it, like the 21-point drop in month one here, means losses are concentrated at onboarding. That puts retention effort in the first month. Read it next to the hazard chart, which shows that cliff directly, and the at-risk count, which says how many customers each point rests on.

⚠️ Common Misread to Avoid: Reading the right-hand end as where every customer ends up. The late points rest on whoever is still being followed, only 41 customers by month 24 here, and the curve stops where the data stops. It is not a forecast: the flat 23% at the end says nothing about whether those customers will stay.

📐 Technical Definition: A customer censored at an age leaves \(n_i\) afterwards without ever counting as churn, so each one contributes for exactly as long as they were observed. This assumes censoring is unrelated to the risk of leaving. The tool's tests check its curve against the lifelines KaplanMeierFitter.

🎯 The Manager's Question: Do these items share enough common ground for a factor analysis to find anything?

📊 Worked Business Example: The default Environmental Consciousness items have an overall KMO of 0.84, in Kaiser's meritorious band. The sports item's own value (its MSA) is 0.40, the only one below 0.50, which singles it out before any factor is extracted.

⚖️ How to Judge the Number: It runs from 0 to 1; the tool warns below 0.60. Read the overall value next to each item's MSA in the table, where an item under 0.50 is a candidate to drop. Read it alongside Bartlett's test, which only checks that some correlation exists.

⚠️ Common Misread to Avoid: Treating KMO as a reliability score. A high KMO says the items share correlation that factors could explain. It says nothing about how many factors there are, or whether a summed score would be consistent; alpha and omega answer that.

📐 Technical Definition: Partial correlations come from the inverse of the correlation matrix. Large partials mean pairs of items are linked only to each other, not through a shared factor, which pulls KMO down. Each item's MSA is the same ratio over that item's row only. The tool matches factor_analyzer's calculate_kmo to five decimals.

🎯 The Manager's Question: Are the model's misses bigger for some kinds of cases than for others?

📊 Worked Business Example: In the B2B deal-size preset the test gives LM = 6.48 on 1 degree of freedom, p = 0.011. The residual spread is 2.5 times wider for the largest third of deals than for the smallest.

⚖️ How to Judge the Number: A p-value below {alpha} says the uneven spread is more than chance. Call it practically uneven only when the widest spread is also at least twice the narrowest, a 2:1 ratio. Read it together with that spread ratio: both together are the tool's warning, and either alone is a watch.

⚠️ Common Misread to Avoid: Thinking uneven spread biases the coefficients. It breaks their standard errors, so the p-values and intervals are off. Robust standard errors or a log of the outcome repair that.

📐 Technical Definition: LM is compared with a chi-square on \(k - 1\) degrees of freedom, where k is the number of coefficients. The Koenker version does not assume normal errors, and matches statsmodels het_breuschpagan with its default robust setting.

🎯 The Manager's Question: Do the groups sit at different places in the overall ranking, or could the gap be chance?

📊 Worked Business Example: Across four channels and 240 sign-ups, H = 62.5 on 3 degrees of freedom, p < 0.001. Referral’s mean rank of 59 sits far below the other three, which are all near 140.

⚖️ How to Judge the Number: Not judged on its own, because it grows with the number of groups. With no difference it averages about k - 1, the degrees of freedom, so 3 here. Read it next to epsilon squared for size and the pairwise table for which groups differ.

⚠️ Common Misread to Avoid: Reading a significant H as "every group differs". It says only that at least one group stands apart from the rest. Here three of the six pairs differ, and all three involve Referral.

📐 Technical Definition: The tool divides H by the usual tie correction and reads it against a chi-square distribution on k - 1 degrees of freedom. The chi-square step is an approximation, so very small groups make it less trustworthy.

🎯 The Manager's Question: Are very large misses more common than a bell curve would predict?

📊 Worked Business Example: For the 60 ad campaigns the excess kurtosis is 0.91, just under the tool's heavy-tail line of 1. The B2B deals-by-industry preset reaches 4.8, because Software deals miss far more widely than the rest.

⚖️ How to Judge the Number: Zero matches a bell curve. Above 1 means heavy tails, with more extreme misses than a bell curve, and the tool calls it severe above 3. Below zero means lighter tails. Read it next to the skewness and the Q-Q plot, which shows whether the tails or one stray case drive it. With heavy tails, check the extreme rows before trusting the p-values.

⚠️ Common Misread to Avoid: Reading it as how pointed the peak looks. It is driven by the tails: a few extreme misses can push it up on their own. In the deals-by-industry preset, the 4.8 comes from Software's wide misses, not from a sharp peak.

📐 Technical Definition: Here \(m_r = \frac{1}{n}\sum(e_i-\bar e)^r\), where \(e_i\) is the residual for case i and \(\bar e\) their mean. This matches scipy.stats.kurtosis with fisher=True and bias=False. Pooling groups with different spreads also produces heavy tails, so the tool asks you to fix unequal spread first.

🎯 The Manager's Question: When the model misses high one period, does it tend to miss high the next period too?

📊 Worked Business Example: In the weekly ad-sales preset the lag-1 autocorrelation of the residuals is 0.67 across 104 weeks. The chance band is only about plus or minus 0.2, so the carry-over is real.

⚖️ How to Judge the Number: 0 means no carry-over. A value further from 0 than about 2 divided by the square root of the number of rows is more than chance. Read it next to Durbin-Watson, which is about 2(1 - r1), and only when rows are in time order.

⚠️ Common Misread to Avoid: Thinking carry-over biases the slope. The slope stays centred, but the standard errors come out too small, so effects look more certain than they are. At 0.67 the true uncertainty is much wider than the coefficient table shows.

📐 Technical Definition: The rough chance band is \(\pm 2/\sqrt{n}\), where n is the number of rows. The tool sorts the residuals by the time column before computing it. Ljung-Box extends the same idea to several lags at once.

🎯 The Manager's Question: Which single touch gets all the credit for this sale, and what does that hide?

📊 Worked Business Example: In the lab every purchase happens inside one session, so a $94 order is credited entirely to that session's source. Email's $49,995 is the sum of such single-session credits, nothing more.

⚖️ How to Judge the Number: There is no right number here; the rule hands 100% of each sale to the final touch. Read it next to the Shapley or Markov share in the attribution labs: a closer such as email or brand search usually shrinks there, an opener grows. Then read it next to the spend and ask whether the buyer would have come anyway.

⚠️ Common Misread to Avoid: Reading email's 20.8x last-click ROAS as proof that email caused those sales. Three sends to 41,000 existing subscribers mostly reach people who already knew the brand, so part of that revenue would have arrived without the email. Last-click cannot say how much.

📐 Technical Definition: The lab's world has exactly one touch per purchase, so every attribution model returns the same credit here. The lab holds attribution fixed so that only the labels can move. Session-scoped reports in GA4, like the ones here, credit the session's source. Its conversion reports can be switched to other models.

🎯 The Manager's Question: What share of the swings in buying comes from who the customer is, rather than week-to-week chance?

📊 Worked Business Example: Thistledown's latent ICC is 0.35: about a third of the variation in shoppers' underlying propensity to buy lies between shoppers. Saltmarsh Books comes in at 0.001, so its readers are nearly interchangeable.

⚖️ How to Judge the Number: 0 means customers barely differ beyond the model's terms; 1 would mean who the customer is explains everything. Saltmarsh's 0.001 is no loyalty to speak of, while Thistledown's 0.35 means loyalty can explain much of the apparent habit. Read it next to the between-customer SD, which it is computed from.

⚠️ Common Misread to Avoid: Reading it as an R-squared for the model. It describes how propensity varies, not how well anything is forecast. Thistledown's 0.35 says nothing about forecast quality; the holdout table does.

📐 Technical Definition: "Latent" because it is computed on the log-odds scale of an assumed underlying propensity, not on the 0/1 purchases. It rests on the logistic noise of the logit model, and the tool shows it only when model C is fitted.

🎯 The Manager's Question: Among training customers who followed the same path, how often did each outcome happen?

📊 Worked Business Example: In the gym preset's default training rows, 31 month-to-month members have 7 or fewer visits a month and 5 or fewer months of membership, and 27 of them churned. Every member on that path gets a churn rate of 87%.

⚖️ How to Judge the Number: The no-model figure is the overall rate, 25% churn in the training rows, and a leaf well above it marks a segment to target. A leaf built from hundreds of customers gives a steady rate; one built from a dozen does not. Read it next to the leaf size and the 95% range.

⚠️ Common Misread to Avoid: Reading the leaf rate as each member's personal chance. Every customer in a leaf gets exactly the same rate, so the tree cannot tell apart two members of one leaf who differ on things it did not split on. A 100% leaf usually means a small leaf, not certainty.

📐 Technical Definition: The tool uses the raw share with no smoothing. Rows in a file the tree is scoring do not change it.

🎯 The Manager's Question: How far could this leaf's rate move just because of which customers happened to land in it?

📊 Worked Business Example: The gym leaf where 27 of 31 training members churned has a rate of 87% and a 95% Wilson range of 71% to 95%. A leaf of 10 members with 8 churners has a rate of 80% but runs from 49% to 94%.

⚖️ How to Judge the Number: A range that spans 50% cannot say whether the leaf leans yes or no. It is narrow when hundreds of training customers reached the leaf and wide when a dozen did. When the range on the predicted category spans more than 40 points, the tool counts the row as coming from a thin leaf and says to treat it as rough. Read it next to the leaf size.

⚠️ Common Misread to Avoid: Reading the range as the whole uncertainty. With a different sample the tree could have split differently and put this customer in another leaf altogether. Treat the range as a minimum.

📐 Technical Definition: This is the Wilson score interval for a proportion. Unlike the plain \(\hat{p} \pm z \cdot SE\), it stays inside 0 to 1. It also does not shrink to nothing for a pure leaf.

🎯 The Manager's Question: How many training customers is this prediction based on?

📊 Worked Business Example: At the gym preset's default 70% split the tree's leaves hold from 31 to 740 training members. The 740-member leaf of two-year members past 5.5 months churns at 5.4% (40 of 740), a rate a new sample would barely move.

⚖️ How to Judge the Number: The bigger, the steadier the rate. At 20 customers or fewer, an evenly split leaf has a 95% range more than 40 points wide, so a leaf at the default minimum of 10 is flagged unless it is lopsided. Read it next to the leaf rate range.

⚠️ Common Misread to Avoid: Reading leaf size as the number of customers in your scored file. It counts training customers. A big leaf gives a steadier rate, not necessarily a right one; it can still rest on the wrong rule.

📐 Technical Definition: The number of training rows that reached this leaf. The minimum leaf size setting, 10 by default, puts a floor under it.

🎯 The Manager's Question: How big a correction does the network make after each batch of customers?

📊 Worked Business Example: At the default 0.01, Ridgeline’s validation loss moves 0.01 or less per epoch and bottoms out at epoch 22. At 0.1 it swings by up to 0.05 (0.577 at epoch 5, 0.529 at epoch 6), so its best epoch, 6, is a lucky dip.

⚖️ How to Judge the Number: 0.01 is the default, and the tool allows 0.0001 to 0.1. Too large makes the validation curve swing by several hundredths between epochs (up to 0.05 at Ridgeline at 0.1). Too small crawls: at 0.0001 Ridgeline is still improving at epoch 300, and the page flags that it never settled. Read it against the shape of the validation curve, not the final score alone.

⚠️ Common Misread to Avoid: A higher rate does not learn more. It takes bigger steps, which can overshoot the low point again and again, and more epochs do not fix that.

📐 Technical Definition: Adam with \(\beta_1 = 0.9\) and \(\beta_2 = 0.999\), the usual defaults, and bias-corrected averages so the first steps are full size. The learning rate sets the overall scale; Adam adjusts each weight’s step around it.

🎯 The Manager's Question: How unusual is this case's mix of predictor values compared with the rest of the data?

📊 Worked Business Example: In the influencer preset, one celebrity partnership reached 450,000 people while the other 50 reached at most 80,000. Its leverage is 0.89, more than ten times the 0.078 cut-off.

⚖️ How to Judge the Number: Leverage runs from 1/n to 1, and the average is the number of coefficients divided by the number of rows. Twice that average counts as high: 2 × 2 / 51 = 0.078 for the influencer preset. It depends only on the predictors, so read it next to the standardized residual to see whether the case also misses. Before acting, refit without the high-leverage case and compare.

⚠️ Common Misread to Avoid: Treating every high-leverage case as a problem. A far-out case that sits on the pattern is harmless and even steadies the slope. It becomes a problem when it also misses, which is what Cook's distance measures.

📐 Technical Definition: With one predictor it simplifies to \(1/n + (x_i - \bar{x})^2 / \sum_j (x_j - \bar{x})^2\), where n is the number of cases: leverage is distance from the average predictor value. Leverages sum to k, the number of coefficients, so the average is k/n and the tool flags values above 2k/n.

🎯 The Manager's Question: How often does this word appear per 1,000 words written in each group?

📊 Worked Business Example: In Harborplace reviews, "lukewarm" appears 18 times across 900 total words. That equals 20.0 per 1,000 tokens (20.0‰).

⚖️ How to Judge the Number: Compare Group A rate per 1,000 tokens against Group B rate alongside the Fightin’ Words z-score.

⚠️ Common Misread to Avoid: Comparing raw word counts when one group has twice as many reviews or much longer reviews. Normalizing per 1,000 tokens puts unequal group volumes on the same footing.

📐 Technical Definition: The table reports raw counts alongside per-mille rates. That lets you judge both absolute evidence volume and normalized usage intensity.

🎯 The Manager's Question: Do the added terms improve the fit by more than chance alone would?

📊 Worked Business Example: Kestrel's model against the intercept-only model gives LR = 191.8 on 3 degrees of freedom, p far below 0.0001. In the lag table a second lag adds only 2.0, p = 0.16, so one lag is enough.

⚖️ How to Judge the Number: An LR of 0 means the added terms change nothing. With one added term, an LR above 3.84 gives p below 0.05. Read the p-value against {alpha}, and read it together with the held-back scores, because a better fit to the past is not always a better forecast.

⚠️ Common Misread to Avoid: Reading a significant LR test as a model that forecasts well. It compares fits on the same past rows and says the added terms carry information there. Whether they help on new periods is what the held-back log loss answers.

📐 Technical Definition: It is compared with a chi-square on the number of added terms. The two models must be fitted to the same rows, and the smaller must sit inside the bigger. Against the intercept-only model it equals the drop in deviance, \(D_{\text{null}} - D_{\text{model}}\), which is how the logistic regression tool prints it.

🎯 The Manager's Question: What has to be true about the model's misses before I can trust its p-values and intervals?

📊 Worked Business Example: The 60 ad campaigns fail L: the misses arch, so the slope of 0.665 mixes high returns at small budgets with low returns at large ones. The tool asks for that fix before it rechecks E and N.

⚖️ How to Judge the Number: L (linearity) matters most, because a missed bend misleads the coefficients themselves. I (independence) is next, then E (equal spread), which breaks the standard errors. N (normality) matters least, especially with 30 or more rows. Read the scorecard together with Cook's distance, since one case can make any check fail.

⚠️ Common Misread to Avoid: Thinking the predictor or the outcome must be bell-shaped. The assumptions are about the errors, so a skewed spend column is fine. Check the residuals, not the raw columns.

📐 Technical Definition: Linearity: \(E(\varepsilon_i \mid \mathbf{x}_i) = 0\). Independence: errors of different cases are unrelated. Equal variance: \(\text{Var}(\varepsilon_i \mid \mathbf{x}_i) = \sigma^2\), where \(\sigma\) is the error standard deviation. Normality: each \(\varepsilon_i\) is normal.

🎯 The Manager's Question: Has the model left a time pattern in its mistakes that another lag could use?

📊 Worked Business Example: Fit Kestrel with no outcome lag and Q is 149.5 on 10 lags, p below 0.0001, so yesterday's surprise predicts today's. Add one lag and Q falls to 6.7, p = 0.67, and the leftover pattern is gone.

⚖️ How to Judge the Number: A p-value below {alpha} says the surprises still resemble earlier surprises, so the model is missing some memory. Add a lag of {outcome} or a missing driver and check again. Above {alpha}, no pattern stands out at the lags tested. Read it next to the autocorrelation bars, which show which lag sticks out.

⚠️ Common Misread to Avoid: Reading p = 0.67 as a guarantee that nothing is left. A pass only says this test found no pattern at the lags it checked, and a pattern at a longer lag can remain.

📐 Technical Definition: Q is compared with a chi-square on \(L\) degrees of freedom, or \(L - p\) when the model already uses \(p\) lags of the outcome. Rows must be in time order. For a yes/no outcome the residuals are Pearson residuals, \((y_t - \hat{p}_t)/\sqrt{\hat{p}_t(1-\hat{p}_t)}\), and the test is approximate, because it was built for numeric residuals.

🎯 The Manager's Question: How strongly does this survey item reflect the idea the factor stands for?

📊 Worked Business Example: In the default one-factor solution, "Support green policies" loads 0.65 and "Watch sports on weekend" loads 0.03. The first is a good marker of environmental concern; the second measures something else entirely.

⚖️ How to Judge the Number: Loadings run from -1 to 1, and 0 means no link to the factor. The tool calls a primary loading below 0.40 weak, and such an item is a candidate to drop or rewrite; the chapter calls 0.70 and up strong. Read it next to the item's communality and its second-highest loading, since an item can load 0.50 and still cross-load.

⚠️ Common Misread to Avoid: Reading a loading of 0.60 as "the item explains 60% of the factor". On a single factor a loading is the item's correlation with the factor, so its square is the share of the item the factor explains. Here that is 0.60 squared, 0.36, about a third.

📐 Technical Definition: The tool extracts factors by principal axis factoring, with principal components as the alternative. It then rotates with Promax, which lets factors correlate. The loadings shown are the pattern matrix: each is a regression weight with the other factors held fixed, not a plain correlation.

🎯 The Manager's Question: Setting the scatter aside, which way do the points drift across the plot?

📊 Worked Business Example: On the ad campaigns' residual plot the LOWESS line sits below 0 at the smallest and largest budgets and above 0 in the middle. That arch is the bend RESET flags with p below 0.001.

⚖️ How to Judge the Number: On a residual plot, a flat line along 0 is good; a bow or a ramp is the pattern to act on. Trust the middle more than the ends, which rest on few points. Read it together with the RESET test, which says whether the bow is more than chance.

⚠️ Common Misread to Avoid: Reading it as a fitted model or a test. It is a reading aid that follows the points wherever they go. A hook at the very edge rests on a handful of cases and is weak evidence.

📐 Technical Definition: At each point the tool fits a weighted straight line to the nearest \(fn\) points, where \(f = 2/3\) is the span and n is the number of points. Three robustness passes then down-weight points with large misses. These are the statsmodels lowess defaults.

🎯 The Manager's Question: How well does the model account for the outcomes it was fitted to?

📊 Worked Business Example: Kestrel's model reaches -247.0 against -342.9 for the overall rate alone, a gain of 95.9. Twice that gain, 191.8, is the model chi-square printed beside it.

⚖️ How to Judge the Number: It is never above 0, and closer to 0 is better. It means something only against another model fitted to the same rows, such as the intercept-only model or one with a lag more. Twice the gap is the likelihood-ratio statistic, so a gain of about 1.9 for one added term is the p = 0.05 bar.

⚠️ Common Misread to Avoid: Reading -247 as a bad score because it is far from 0. Its size grows with the number of rows, so a model fitted to 727 days sits further below 0 than one fitted to 100. Only the difference between models fitted to the same rows carries meaning.

📐 Technical Definition: The fit picks the coefficients that make this as large as possible, and minus twice it is the deviance. Models with a numeric outcome, such as a time-series regression, use the same idea with their own likelihood.

🎯 The Manager's Question: How costly, on average, were the forecast probabilities once we saw what actually happened?

📊 Worked Business Example: On Kestrel's 56 held-back days the model scores 0.313, against 0.418 for momentum only and 0.497 for the base rate. The drivers cut the penalty by 25% beyond momentum, so promotions and truck days carry real forecasting information.

⚖️ How to Judge the Number: Lower is better and 0 is perfect, but there is no universal good value. Always forecasting 50% scores 0.693, and always forecasting the base rate is the bar any forecast has to beat. Read it against "momentum only" on the same held-back periods: the gap is what the drivers add.

⚠️ Common Misread to Avoid: Reading 0.31 as a 31% error rate. It is an average penalty, not a percentage, and a confident miss costs far more than a hesitant one. One forecast of 99% for a day that turns out "no" adds 4.6 on its own, about fifteen times Kestrel's average.

📐 Technical Definition: Each forecast comes from a model refitted on the periods before it only, so no forecast has seen its own outcome. Probabilities are clipped to \([10^{-15}, 1 - 10^{-15}]\), so a 0% or 100% forecast scores a large but finite penalty. It is the quantity logistic regression minimizes when it is fitted, scored here on periods it was not fitted to. Dynamic Logistic Regression's held-back scores are checked against scikit-learn's log_loss in its reference tests.

🎯 The Manager's Question: Does this driver push the chance of a "yes" up or down, and how hard?

📊 Worked Business Example: Kestrel's delivery-day coefficient is -2.16: a truck day pushes the log-odds of an empty shelf down by 2.16. As an odds ratio that is about 0.12, so a truck day cuts the odds by close to 90%.

⚖️ How to Judge the Number: 0 means no association: above 0 pushes toward "yes", below 0 toward "no". The scale is hard to read directly, so translate it: 0.69 doubles the odds and -0.69 halves them. Read it next to its odds ratio and the percentage-point effects, which are the numbers a decision needs.

⚠️ Common Misread to Avoid: Reading a coefficient of 0.5 as "50 points more likely". It is a change in log-odds, which multiplies the odds by about 1.65. What that does to the probability depends on where it starts: from 10% it reaches about 15%, from 50% about 62%.

📐 Technical Definition: Each \(\beta\) is the change in log-odds for a one-unit step in its predictor, the others held fixed, and \(e^{\beta}\) is the odds ratio. For a categorical predictor the step is from the reference level to that level. The fit is by maximum likelihood; Dynamic Logistic Regression's reference tests check its estimates against statsmodels Logit to six decimals.

🎯 The Manager's Question: How much higher are the smoothed odds of seeing a word in Group A than in Group B?

📊 Worked Business Example: When Harborplace uses "waitress" 28 times and Downtown uses it 4 times, the smoothed log-odds gap is +1.72. Positive values favor Group A.

⚖️ How to Judge the Number: A value of 0.00 means equal usage rates. Read the raw magnitude next to the z-score and word count.

⚠️ Common Misread to Avoid: Reading a log-odds difference of 1.0 as a 100% point jump in frequency. Because it sits on the natural log scale, +1.0 multiplies the odds of that word by about 2.72.

📐 Technical Definition: Each group log-odds is computed after adding prior pseudocounts \(\alpha_w = \alpha_0 (y_w / N)\) to the observed token count.

🎯 The Manager's Question: Do these customer groups leave on different schedules, or could the gap be chance?

📊 Worked Business Example: Split Parcel & Pantry by acquisition channel and the test gives chi-square 353.3 on 3 degrees of freedom, p < 0.001. Promotion sign-ups churned 481 times where 243 were expected if every channel shared one curve.

⚖️ How to Judge the Number: Judge the p-value against {alpha}: below it, the schedules differ by more than chance. The test says whether the timing differs, never by how much. Read it next to the segment curves and their medians, which carry the size: 3 months for Promotion against 12 to 13 for the other channels.

⚠️ Common Misread to Avoid: Reading a non-significant result as "these groups behave alike". The test is weak when the curves cross, because an early gap and a late gap in opposite directions cancel out. Check the curves, and compare the groups at the age you care about.

📐 Technical Definition: At each churn age, group g expects \(d_t\, n_{gt} / n_t\) of the \(d_t\) events: its share of the customers still at risk. With two groups the statistic reduces to \((O_1 - E_1)^2 / V_{11}\). With three or more the tool keeps the covariance between groups and leaves one group out, because the deviations sum to zero; this matches the multivariate log-rank test in lifelines. Groups under 10 customers are left out.

🎯 The Manager's Question: Does this predictor push the log event rate up or down, and by how much?

📊 Worked Business Example: In CraveDrop, the membership slope is +0.489 on the log-count scale. Exponentiating gives exp(0.489) = 1.63, meaning membership multiplies expected quarterly orders by 1.63 (+63%).

⚖️ How to Judge the Number: 0.00 means no association with the event rate: positive values raise the count rate and negative values lower it. Read it next to its incidence rate ratio and p-value to judge practical and statistical strength.

⚠️ Common Misread to Avoid: Reading a log-rate slope of 0.49 as 0.49 extra events per unit of X. The coefficient is additive on the natural log scale, so you must exponentiate it into an IRR before multiplying the baseline count.

📐 Technical Definition: A one-unit step in \(X_j\) shifts \(\ln(\hat{\mu})\) by \(\hat{\beta}_j\) while holding other predictors and exposure constant. For a categorical predictor, \(\hat{\beta}_j\) compares that category against the reference level.

🎯 The Manager's Question: How much of the mismatch between predicted chances and what customers did do the predictors remove?

📊 Worked Business Example: In the influencer preset the null deviance is 1,380.8 and the residual deviance 1,064.3. The predictors remove 316.5, which is the model chi-square on 7 degrees of freedom.

⚖️ How to Judge the Number: The null deviance gives every row the overall rate, 46.3% here; the residual deviance uses the predictors. The drop between them is tested against a chi-square with one degree of freedom per added term, which is the likelihood ratio test. Read it next to McFadden R², which is 1 minus residual over null, 0.229 here.

⚠️ Common Misread to Avoid: Reading it as a count of wrong predictions or a percentage. It is a sum over rows, so it grows with the sample: 1,064 on 1,000 rows is not a bad fit by itself. Compare deviances only between models fitted to the same rows.

📐 Technical Definition: For 0/1 rows the saturated model fits perfectly, so the deviance equals -2 times the log-likelihood. The null deviance sets every \(\hat{p}_i\) to the overall rate. The tool reports both on the fitting rows.

🎯 The Manager's Question: If this predictor rose 1%, how many points would a typical customer's chance move?

📊 Worked Business Example: In the influencer preset the chance at the mean profile is 52.8%, and swipe-ups have an elasticity of 0.48. A 1% rise from the average 933 swipe-ups, about 9 more, adds roughly 0.48 percentage points to the chance of a sale.

⚖️ How to Judge the Number: Zero means the predictor does not move the chance at that profile. The tool reads the figure as percentage points per 1% rise, with numeric predictors at their means and categories at their sample shares. 0.48 means a 10% rise, about 93 swipe-ups, adds roughly 4.8 points. If you ticked Standardize, every figure shows as 0 because the standardized means are 0; untick it to read this table. Read it next to the odds ratio, and expect a smaller effect for customers far from 50%.

⚠️ Common Misread to Avoid: Reading 0.48 as a 0.48% proportional change in the chance. The tool's figure is in percentage points, so 52.8% becomes about 53.3%. It also holds only near the mean profile; at a 95% chance the same 1% rise moves far less.

📐 Technical Definition: Because \(\partial p/\partial x = \beta p(1-p)\), a 1% rise in \(x\) moves \(p\) by \(0.01\,\varepsilon_j\), which is \(\varepsilon_j\) percentage points: strictly a semi-elasticity. A proportional elasticity would divide by \(\bar{p}\). Category predictors are excluded.

🎯 The Manager's Question: How does the model turn a customer's characteristics into a chance of the outcome?

📊 Worked Business Example: In the influencer preset, a Micro-tier US post with average views and 1,500 swipe-ups scores 1.72 on the log-odds scale, which the S-curve turns into an 85% chance of a sale. At 500 swipe-ups the score is -0.33 and the chance 42%.

⚖️ How to Judge the Number: A score of 0 is a 50% chance, the coin-flip point. The same step in the score moves the chance most near 50% and least near the ends: a score of 2 is 88%, a score of 4 only 98%. Read it next to the odds ratios, which say how far each predictor moves the score, and plan budgets on the probability.

⚠️ Common Misread to Avoid: Reading a coefficient as a fixed change in probability. A slope of 0.5 adds 0.5 to the score, which lifts a 50% customer to 62% but a 95% customer only to 97%. The same predictor moves the chance a lot mid-curve and hardly at all near the ends.

📐 Technical Definition: The score is \(\eta_i = \beta_0 + \beta_1 X_{1i} + \dots + \beta_p X_{pi}\), and the same model on the log-odds scale is \(\log[p_i/(1-p_i)] = \eta_i\). The tool fits the coefficients by maximum likelihood. It assumes independent rows and log-odds that are linear in each numeric predictor.

🎯 The Manager's Question: How much lower could our log-loss go by moving the curve to the best fit?

📊 Worked Business Example: On Email Conversion a curve at 0.339 sits 0.216 above the best curve's 0.123. That is 40% of the 0.541 the predictor can buy, so the curve has captured the other 60%.

⚖️ How to Judge the Number: 0 means you matched the best curve the sliders allow. Its size means something only against the possible improvement, the base-rate guess minus the best curve: 0.541 on Email Conversion but 0.006 on Luxury Test Drive. Read it next to Improvement captured, which divides by that distance.

⚠️ Common Misread to Avoid: Reading a small gap as a near-perfect fit. On Luxury Test Drive a nearly flat curve sits only 0.005 above the best one, yet it captures just 18% of the possible improvement. The best curve beats the flat 10% guess by only 0.006 there, so 0.005 is most of the distance.

📐 Technical Definition: The page prints it to four decimals. The best curve is refitted when the category toggle changes, so the gap always compares curves with the same terms. Dividing it by \(\text{LL}_0 - \text{LL}_{\text{best}}\) and subtracting from 1 gives the improvement captured.

🎯 The Manager's Question: How much of what the predictor can buy over a flat guess does our curve actually deliver?

📊 Worked Business Example: On Email Conversion a curve scoring 0.339 captures 60%: (0.664 - 0.339) / (0.664 - 0.123). The flat 62% guess captures 0% and the best curve 100%.

⚖️ How to Judge the Number: 0% is no better than giving everyone the base rate, and 100% matches the best curve the sliders allow. Below 0% is worse than that flat guess. The lab grades the whole percent: 98% or more is Outstanding, 90% Excellent, 60% Good and 25% Some signal. Read it next to the AUC: a curve can rank the rows like the best one and still capture little.

⚠️ Common Misread to Avoid: Reading 100% as a perfect model. It only means your curve matched the best one the sliders allow. On Ad Click that best curve still scores 0.492, the highest floor of the five scenarios, so 100% there says position alone has been used up, not that clicks are predictable.

📐 Technical Definition: The best curve is found inside the slider ranges by a Nelder-Mead search polished with Newton steps. Log-loss is convex, so that is the bounded maximum-likelihood fit, and the reference tests match it to a bounded scipy fit. The printed share is capped at 100%, and no share is shown when the best curve cannot beat the flat guess.

🎯 The Manager's Question: If this plan were kept up for good, what share of periods would end up "yes"?

📊 Worked Business Example: With trucks on Monday and Thursday, Kestrel's long-run rate is 17.1% of days with an empty shelf. Adding a Saturday drop takes it to 13.5%, about 13 fewer empty-shelf days a year to weigh against $180 a week.

⚖️ How to Judge the Number: Compare a plan against the baseline rate, not against 0: the gap is the plan's lasting effect. A gap of 3.5 points over 365 days is about 13 days a year, the number a recurring cost is weighed against. Read it next to the immediate effects, which show where the change comes from.

⚠️ Common Misread to Avoid: Reading 17.1% as next week's forecast. Next week starts from today's state, so after an empty shelf the forecast runs higher for several days. The long-run rate is where the forecasts settle once that memory fades, assuming the fitted relationships hold.

📐 Technical Definition: The display line is the one-lag, constant-plan case. In general the model makes the last \(p\) outcomes a Markov chain with \(2^p\) states. Over one full cycle of the plan, the product of the one-period transition matrices has a stationary distribution \(\pi\) solving \(\pi M = \pi\), and the long-run rate averages \(\hat{P}(y = 1)\) across that cycle from \(\pi\). It is not defined when the model has a trend term.

🎯 The Manager's Question: What is the longest unbroken streak of identical scale ratings this respondent clicked in a row?

📊 Worked Business Example: On SignalPath's nine-item feature grid, one respondent answers 4 followed by eight consecutive 3s. That gives a Longstring of 8 out of 9 items (88.9% streak ratio).

⚖️ How to Judge the Number: Compare the streak length L_i against the total items m in the block. Streaks covering 80% or more of a long grid (m >= 8) warrant inspection alongside completion speed.

⚠️ Common Misread to Avoid: Treating a streak of four identical answers on a short four-item scale as equivalent to an eight-item streak on a ten-item grid. Always scale the streak by block length and polarity mix.

📐 Technical Definition: Longstring captures partial straightlining when a respondent varies the first answer and then clicks straight down the remaining rows. Dividing \(L_i\) by \(m\) yields the block streak share.

🎯 The Manager's Question: Setting the scatter aside, which way do the points drift across the plot?

📊 Worked Business Example: On the ad campaigns' residual plot the LOWESS line sits below 0 at the smallest and largest budgets and above 0 in the middle. That arch is the bend RESET flags with p below 0.001.

⚖️ How to Judge the Number: On a residual plot, a flat line along 0 is good; a bow or a ramp is the pattern to act on. Trust the middle more than the ends, which rest on few points. Read it together with the RESET test, which says whether the bow is more than chance.

⚠️ Common Misread to Avoid: Reading it as a fitted model or a test. It is a reading aid that follows the points wherever they go. A hook at the very edge rests on a handful of cases and is weak evidence.

📐 Technical Definition: At each point the tool fits a weighted straight line to the nearest \(fn\) points, where \(f = 2/3\) is the span and n is the number of points. Three robustness passes then down-weight points with large misses. These are the statsmodels lowess defaults.

🎯 The Manager's Question: Does Negative Binomial fit significantly better than Poisson on these counts?

📊 Worked Business Example: In CraveDrop, log-likelihood rises from -1,490.2 under Poisson to -1,198.4 under NB2, giving G2 = 583.6 (p < 0.0001). That decisively rejects Poisson equidispersion in favor of Negative Binomial.

⚖️ How to Judge the Number: A p-value below {alpha} rejects Poisson equidispersion (alpha = 0) and favors Negative Binomial. Read it next to the Pearson dispersion ratio and AIC.

⚠️ Common Misread to Avoid: Using a standard two-sided chi-square table without halving the p-value. Because alpha cannot be negative, the null hypothesis alpha = 0 sits on the boundary of the parameter space.

📐 Technical Definition: Under the boundary null \(\alpha = 0\), \(G^2\) follows a 50:50 mixture of a point mass at zero and a \(\chi^2_1\) distribution. For \(G^2 > 0\), the boundary p-value equals \(0.5 \times P(\chi^2_1 > G^2)\).

🎯 The Manager's Question: Does the model do well on every category, including the rare ones?

📊 Worked Business Example: On the 1,050 test rows of the FitLife churn preset, the default tree scores F1 0.85 for Stayed but 0.27 for Churned, a macro F1 of 0.56. Its 74.9% accuracy hides that weakness.

⚖️ How to Judge the Number: It runs from 0 to 1, higher is better, and every category counts equally. Always guessing the most common category scores 0 for every other one, so about 0.43 for a 75:25 split. Read it next to accuracy and the per-category rows.

⚠️ Common Misread to Avoid: Reading it as another accuracy. A low value beside a respectable accuracy usually means one category is hardly ever predicted. Check the per-category rows.

📐 Technical Definition: A category that occurs but is never predicted correctly scores 0, as scikit-learn does with zero_division set to 0. So a model that ignores a small category cannot earn a perfect score.

🎯 The Manager's Question: On average, how far off is a prediction, in {outcome}'s own units?

📊 Worked Business Example: In the EDM event-spend preset the model misses a patron's spend by $8.80 on average. Its RMSE of $11.0 is 1.25 times that, the ratio expected when misses are bell-shaped.

⚖️ How to Judge the Number: Lower is better and 0 is perfect. RMSE is always at least as big as MAE, and about 1.25 times it when misses are bell-shaped. Read it next to RMSE: a ratio well above 1.25 means a handful of rows are being missed badly.

⚠️ Common Misread to Avoid: Reading MAE as a promise that every prediction lands within it. When misses are bell-shaped, about four predictions in ten miss by more. Use a prediction interval for a range around one customer.

📐 Technical Definition: Every miss counts in proportion to its size. For a transformed outcome the predictions are converted back first, as with RMSE.

🎯 The Manager's Question: Which constructs are both widespread in customer reviews and heavily damaging or helpful to {outcome}?

📊 Worked Business Example: In Canyon Creek Bistro, Service Delay has 36% prevalence and a -1.68 star slope. That places it in the bottom-right urgent-fix quadrant.

⚖️ How to Judge the Number: The horizontal line at 0.00 separates positive and negative slopes, read alongside the vertical line at mean prevalence.

⚠️ Common Misread to Avoid: Chasing a loud complaint in the bottom-left quadrant before fixing a bottom-right driver. A high-prevalence negative driver costs far more total rating points across the customer base.

📐 Technical Definition: Plotting prevalence against partial OLS slopes guides resource allocation. It stops teams from over-reacting to frequent harmless chatter or rare incidents.

🎯 The Manager's Question: How many head-to-head pairings does {groupB} win against {groupA}, and is that more than half?

📊 Worked Business Example: In the whale-order case, U = 5,504 for Flow B out of a possible 95 × 90 = 8,550 pairings. Dividing gives 0.64, the probability of superiority.

⚖️ How to Judge the Number: Not judged on its own, because its size depends on the group sizes. The no-difference anchor is half the pairings, 4,275 here. Report it so a reader can reproduce the test, and read it next to the probability of superiority for the size of the gap.

⚠️ Common Misread to Avoid: Reading a bigger U as a bigger difference. U cannot be compared across studies, because its maximum is the product of the two group sizes. Divide by that product before comparing, which gives the probability of superiority.

📐 Technical Definition: The p-value is exact when both groups have 20 or fewer cases and no ties. Otherwise it uses the normal approximation with tie and continuity corrections. The tool is checked against SciPy’s mannwhitneyu.

🎯 The Manager's Question: How many points of margin have discounts cost the business compared with selling at list price?

📊 Worked Business Example: NordicThreads starts at 5.1 points of leakage, from the 20% of shoppers already buying on deal. At the default 40% depth it reaches 36.7 points by Year 18, leaving an 8.3% margin.

⚖️ How to Judge the Number: 0 points means everyone who buys pays list. The card counts down from the 45% list-price margin, so 45 points means deal sales only break even and more than 45 means they lose money. Read it next to the average fair price on the same card: below $45, full-price shoppers start waiting.

⚠️ Common Misread to Avoid: Reading leakage as the cost of the codes that customers redeemed. Most of the 36.7 points come from former full-price shoppers who saw neighbors pay $30 and switched. The promotion reaches further than its redemptions.

📐 Technical Definition: Both terms are percentages of revenue. The 45 is the list-price margin, from a unit cost of 55% of list.

🎯 The Manager's Question: If we add one more dollar to this channel next week, how many sales dollars come back?

📊 Worked Business Example: In the default Brightwell model the next dollar on TV returns $1.63 and the next dollar on search $1.24, though search's average ROI is twice TV's. That is why the reallocation step moves money from search into TV.

⚖️ How to Judge the Number: 1.00 is the sales break-even for the next dollar; profit needs about 1 divided by your gross margin. The tool marks a channel "Room to Grow" above 1.20 when that is at least 60% of its average ROI. Read it next to average ROI and across channels: money should move until the next dollar returns about the same everywhere.

⚠️ Common Misread to Avoid: Treating marginal ROI as fixed, so that doubling the budget doubles the return. It falls as spend rises: moving TV from $13,600 to $16,000 a week lowers its marginal ROI from $1.63 to $1.44. That fall, together with the shift limits, keeps the optimizer from putting everything into one channel.

📐 Technical Definition: This is the slope of the steady-state response curve \(Y_c(x) = \beta_c\, x / (x + K_c(1 - \lambda_c))\), the weekly contribution if spend held at x every week. The tool evaluates it at each channel's average weekly spend and checks it against a finite difference.

🎯 The Manager's Question: How much ad spend does each credited purchase cost us?

📊 Worked Business Example: Plan A's $10,000 buys 2,000,000 impressions, 10,000 clicks and 200 purchases, so CPA is $10,000 ÷ 200 = $50. Halve the purchase rate to 1% and CPA doubles to $100.

⚖️ How to Judge the Number: Judge it against what a purchase is worth: a $50 CPA pays off only if each purchase earns more than $50 in gross margin. With zero purchases it cannot be computed. Read it next to the click rate and the purchase rate, which show where in the funnel the cost comes from.

⚠️ Common Misread to Avoid: Reading a $50 CPA as $50 per sale the ads created. It counts credited purchases, and some of those buyers would have bought anyway; only a holdout test measures the extra sales.

📐 Technical Definition: The tool computes it from unrounded expected counts and shows no value, rather than an infinite one, when expected purchases are 0. Spend is media only; creative, fees and product margin are left out.

🎯 The Manager's Question: Do these items hang together once strong and weak items are allowed to count differently?

📊 Worked Business Example: On the default Environmental Consciousness items, omega is 0.72 while alpha is 0.69. Omega weights each item by its loading, so the near-zero sports item drags it down less.

⚖️ How to Judge the Number: Read it on the same bands as alpha, with 0.70 as the usual line. Read it next to alpha: a gap of a few hundredths is normal, and a gap of 0.05 or more says the items are unequal, so report omega. The whole-scale omega assumes one factor, so check that parallel analysis keeps one first.

⚠️ Common Misread to Avoid: Treating omega and alpha as rival verdicts and quoting the higher one. Omega is alpha without the equal-weights assumption, so when they differ, omega is the better estimate of the same thing. On the default data they agree: 0.72 and 0.69 both sit near the 0.70 line.

📐 Technical Definition: The tool fits one factor by principal axis factoring and puts its standardized loadings into the formula, so omega assumes a single factor and uncorrelated errors. For a scale with several factors, use the per-factor omega in the validity table instead. On one factor's items it is the same number as composite reliability.

🎯 The Manager's Question: Do these items hang together once strong and weak items are allowed to count differently?

📊 Worked Business Example: On the default Environmental Consciousness items, omega is 0.72 while alpha is 0.69. Omega weights each item by its loading, so the near-zero sports item drags it down less.

⚖️ How to Judge the Number: Read it on the same bands as alpha, with 0.70 as the usual line. Read it next to alpha: a gap of a few hundredths is normal, and a gap of 0.05 or more says the items are unequal, so report omega. The whole-scale omega assumes one factor, so check that parallel analysis keeps one first.

⚠️ Common Misread to Avoid: Treating omega and alpha as rival verdicts and quoting the higher one. Omega is alpha without the equal-weights assumption, so when they differ, omega is the better estimate of the same thing. On the default data they agree: 0.72 and 0.69 both sit near the 0.70 line.

📐 Technical Definition: The tool fits one factor by principal axis factoring and puts its standardized loadings into the formula, so omega assumes a single factor and uncorrelated errors. For a scale with several factors, use the per-factor omega in the validity table instead. On one factor's items it is the same number as composite reliability.

🎯 The Manager's Question: How much better does the model fit than knowing only the overall rate?

📊 Worked Business Example: Kestrel's model has a log-likelihood of -247.0 against -342.9 for the overall rate alone, so McFadden R-squared is 1 - 247.0 / 342.9 = 0.28. That is a good fit for a logistic model.

⚖️ How to Judge the Number: 0 means no better than the overall rate, and 1 would be a perfect fit. Values from 0.2 to 0.4 already indicate a good fit for a logistic model. Read it with the held-back scores, because it is an in-sample fit and says nothing about forecasting new periods.

⚠️ Common Misread to Avoid: Reading 0.28 as "the model explains 28% of the variation". It is a ratio of log-likelihoods, and it runs much lower than a linear regression's R-squared for an equally useful model.

📐 Technical Definition: Adding any term can only raise it in sample, so it rewards complexity; AIC and BIC charge for that. Dynamic Logistic Regression's reference tests check its value against statsmodels' prsquared.

🎯 The Manager's Question: Did significantly more matched pairs switch in one direction than the other?

📊 Worked Business Example: Suppose 60 customers switched from No to Yes and 20 switched from Yes to No. McNemar tests that 60-to-20 split against 50/50.

⚖️ How to Judge the Number: Read the chi-square statistic alongside its p-value against {alpha} and the switcher count (b + c).

⚠️ Common Misread to Avoid: Thinking people who stayed Yes-Yes or No-No affect the test. McNemar depends only on the discordant switcher cells b and c.

📐 Technical Definition: With continuity correction, the numerator uses \((|b - c| - 1)^2\) to better approximate the exact binomial tail when \(b + c\) is small.

🎯 The Manager's Question: Did significantly more matched pairs switch in one direction than the other?

📊 Worked Business Example: Suppose 60 customers switched from No to Yes and 20 switched from Yes to No. McNemar tests that 60-to-20 split against 50/50.

⚖️ How to Judge the Number: Read the chi-square statistic alongside its p-value against {alpha} and the switcher count (b + c).

⚠️ Common Misread to Avoid: Thinking people who stayed Yes-Yes or No-No affect the test. McNemar depends only on the discordant switcher cells b and c.

📐 Technical Definition: With continuity correction, the numerator uses \((|b - c| - 1)^2\) to better approximate the exact binomial tail when \(b + c\) is small.

🎯 The Manager's Question: Did significantly more matched pairs switch in one direction than the other?

📊 Worked Business Example: Suppose 60 customers switched from No to Yes and 20 switched from Yes to No. McNemar tests that 60-to-20 split against 50/50.

⚖️ How to Judge the Number: Read the chi-square statistic alongside its p-value against {alpha} and the switcher count (b + c).

⚠️ Common Misread to Avoid: Thinking people who stayed Yes-Yes or No-No affect the test. McNemar depends only on the discordant switcher cells b and c.

📐 Technical Definition: With continuity correction, the numerator uses \((|b - c| - 1)^2\) to better approximate the exact binomial tail when \(b + c\) is small.

🎯 The Manager's Question: What does each customer bring in on average, the figure that scales up to a total?

📊 Worked Business Example: Flow A's 95 orders average $113.38, but without its single $4,120 bulk order the other 94 average $70.76. One case moves the average by more than $40.

⚖️ How to Judge the Number: It is the balance point, where the deviations above and below sum to 0. Read it next to the median: a mean a quarter or more above the median (1.25 times), like Flow A's $113 against $63, is a few big values pulling it up. It is the right number for totals and budgets, since total revenue is the mean times the count.

⚠️ Common Misread to Avoid: Reading the mean as what a typical customer delivers. In skewed data such as customer lifetime value, most cases fall below it: 84 of Flow A's 95 orders sit under its $113 mean. Yet for untargeted acquisition the mean is the expected value E[X] per customer you sign. So capping acquisition cost (CAC) at the median rather than the mean leaves profitable expected return on the table.

📐 Technical Definition: For 0/1 data, such as whether a click converted, \(\bar{x} = \frac{1}{n}\sum x_i\) is the conversion rate: Aura's 70 sales in 1,850 clicks average 0.038. The mean has no resistance to outliers, so one arbitrarily large value moves it arbitrarily far.

🎯 The Manager's Question: How well would predicting the average for everyone do?

📊 Worked Business Example: At Wrenfield, predicting the training average for all 500 held-back markets misses by $31,000 of quarterly revenue (RMSE). The regression cuts that to $12,800 and the network to $12,400, so most of the gain comes from having any model at all.

⚖️ How to Judge the Number: It is the floor every model must clear: a holdout R² of 0 means a model does no better than this guess. Read the models against it, because the gap is what modeling adds. At Wrenfield the regression already closes $18,200 of the $31,000, so the network’s extra $430 is the part to check with the paired bootstrap interval.

⚠️ Common Misread to Avoid: Beating it does not mean a model is good. Almost any model with one useful input clears it; the real test is whether it beats the regression you could explain.

📐 Technical Definition: The training mean is predicted for every held-back row and scored the same way as the models. Its RMSE is therefore about the outcome’s standard deviation in the holdout. Using the training mean, not the holdout’s own, keeps the guess honest: it knows nothing about the rows it is scored on.

🎯 The Manager's Question: How big is the difference, in a form I can compare across tests and studies?

📊 Worked Business Example: Aura's 3.8% against the 2.4% baseline is a Cohen's h of 0.08 at 1,850 clicks. Big Data Mode's 185,000 clicks push the p-value to almost zero, and h stays at 0.08.

⚖️ How to Judge the Number: 0 means no difference. Each family has its own scale; for Cohen's d and h the report panels call below 0.2 negligible, 0.2 small, 0.5 moderate and 0.8 large (Cohen, 1988). Read it next to the p-value, then translate it into money: Aura's h of 0.08 is negligible on that scale, yet 1.4 extra points on 200,000 clicks is worth about $74,000 a year.

⚠️ Common Misread to Avoid: Reading a tiny p-value as a big effect. Big Data Mode shrinks Aura's p-value from 0.00005 to almost zero while the lift stays 1.4 points and h stays 0.08. Sample size moves the p-value; only the effect size says how much.

📐 Technical Definition: A magnitude scaled so it does not grow with the sample size. Standardized versions also drop the units, so a click test and a survey can be compared. The one-sample tool reports Cohen's h for a proportion and Cohen's d for a mean. h rates a large relative lift near a low base rate as small (Aura's 58% lift is h 0.08), which is why the tool also states the lift in points and dollars. The bands are conventions, not laws, so a number is only small or large on a named scale.

🎯 The Manager's Question: How big is the difference, in a form I can compare across tests and studies?

📊 Worked Business Example: Aura's 3.8% against the 2.4% baseline is a Cohen's h of 0.08 at 1,850 clicks. Big Data Mode's 185,000 clicks push the p-value to almost zero, and h stays at 0.08.

⚖️ How to Judge the Number: 0 means no difference. Each family has its own scale; for Cohen's d and h the report panels call below 0.2 negligible, 0.2 small, 0.5 moderate and 0.8 large (Cohen, 1988). Read it next to the p-value, then translate it into money: Aura's h of 0.08 is negligible on that scale, yet 1.4 extra points on 200,000 clicks is worth about $74,000 a year.

⚠️ Common Misread to Avoid: Reading a tiny p-value as a big effect. Big Data Mode shrinks Aura's p-value from 0.00005 to almost zero while the lift stays 1.4 points and h stays 0.08. Sample size moves the p-value; only the effect size says how much.

📐 Technical Definition: A magnitude scaled so it does not grow with the sample size. Standardized versions also drop the units, so a click test and a survey can be compared. The one-sample tool reports Cohen's h for a proportion and Cohen's d for a mean. h rates a large relative lift near a low base rate as small (Aura's 58% lift is h 0.08), which is why the tool also states the lift in points and dollars. The bands are conventions, not laws, so a number is only small or large on a named scale.

🎯 The Manager's Question: Where does each group sit when every case is lined up from lowest to highest?

📊 Worked Business Example: Rank all 185 orders in the whale-order case and Flow A averages rank 80.1 while Flow B averages 106.7. Flow B’s orders tend to sit higher, even though Flow A has the larger mean order value.

⚖️ How to Judge the Number: With no difference every group sits near (N + 1) / 2, which is 93 for 185 orders. A gap from that anchor shows which way a group leans. Read it next to the medians, which say the same thing in the original units.

⚠️ Common Misread to Avoid: Reading a mean rank of 106.7 as 106.70 dollars. It is not a score on your original scale; it is an average position. It also shifts when cases are added to or removed from any group, including the other ones.

📐 Technical Definition: Ranks run from 1 to N across all groups combined. Mean ranks feed both the Kruskal-Wallis H and the Dunn pairwise z-tests.

🎯 The Manager's Question: Do the predictors, taken together, predict {outcome} better than just using its average?

📊 Worked Business Example: For the 60 ad campaigns F is about 378 on 1 and 58 degrees of freedom, with p below 0.001. Spend clearly beats the average, which says nothing yet about whether a straight line is the right shape.

⚖️ How to Judge the Number: If the predictors explained nothing, F would sit near 1. A p-value below {alpha} says they beat the average together; it does not say which one matters. Read it next to the individual coefficient tests and the diagnostics.

⚠️ Common Misread to Avoid: Reading a significant F as "the model is good". It only rules out the idea that none of the predictors matter. The ad campaigns pass it easily and still fail the linearity check.

📐 Technical Definition: Here \(SS_{\text{res}} = \sum e_i^2\), where \(e_i\) is the residual for case i. F is compared with an F distribution on \(k-1\) and \(n-k\) degrees of freedom, and assumes normal errors with equal spread.

🎯 The Manager's Question: How many sales did each channel add on top of what would have sold anyway?

📊 Worked Business Example: In the default Brightwell model, TV contributes about $3.44 million over 104 weeks and search $3.04 million, 49% and 43% of the media total. Display contributes $0: its fitted effect came out negative, so the tool fixed it at zero.

⚖️ How to Judge the Number: 0 means the channel added nothing the model can see. Read each contribution next to what it cost: TV's $3.44 million came from $1.41 million of spend, which is its average ROI. A big contribution can still be a poor home for the next dollar, so read marginal ROI before moving money.

⚠️ Common Misread to Avoid: Treating contribution as counted sales, the way a coupon code is. It is the model's estimate of what sales would lose without the channel. When two channels' spend moves together, the model can split their credit almost arbitrarily, so check the collinearity warning first.

📐 Technical Definition: Media coefficients are fitted by least squares on the training weeks with a non-negativity rule: a channel whose coefficient comes out negative is refitted at zero and flagged. Contributions are summed over every week, holdout included. The numbers match a Python statsmodels reference on all four cases.

🎯 The Manager's Question: What does a typical case look like, setting the extremes aside?

📊 Worked Business Example: In the whale-order case, Flow A's median order is $62.80 and Flow B's is $89.39. Taking out Flow A's single $4,120 bulk order moves its median by 2 cents.

⚖️ How to Judge the Number: Half the cases sit below it and half above. Read it next to the mean: a mean a quarter or more above the median (1.25 times), like Flow A's $113 against $63, means high-spending whales pull it up. Quote the median for a typical customer, while expected-value metrics across an entire acquired cohort depend on the mean.

⚠️ Common Misread to Avoid: Reading the median as the midpoint between the smallest and largest values. Flow A's orders run from $13.54 to $4,120, a midpoint of about $2,067, yet its median is $62.80. It is the middle case once the values are sorted, so a bigger largest order would not move it.

📐 Technical Definition: The 50th percentile. The tool sorts the values and interpolates between neighbours, so with an even count it is the average of the two middle values, as in Excel's MEDIAN. A rank-based test compares where whole distributions sit, not the two medians alone.

🎯 The Manager's Question: By what age has half of a group of customers left?

📊 Worked Business Example: Parcel & Pantry's curve first falls to 49.4% at month 8, so half its customers are gone by month 8, with an interval of 8 to 9. Promotion sign-ups reach that point at month 3.

⚖️ How to Judge the Number: The anchor is the 50% line: the median is the first age at which the survival curve touches it. Compare it against the months a customer needs to repay their acquisition cost; a median of 8 against a 10-month payback means the typical customer leaves before paying back. Read it next to average time retained, which exists even when the median does not.

⚠️ Common Misread to Avoid: Quoting a median when the curve never reaches 50%. If more than half are still active at the last age observed there is no median, and the tool says so instead of inventing one. Any number quoted then has been extrapolated beyond the data.

📐 Technical Definition: It is read off the step curve, so it is always a whole number of periods. The interval comes from where the lower and upper edges of the confidence band cross 0.5, so it is asymmetric and can be open on the right.

🎯 The Manager's Question: Is this site built for readers, or built mainly to sell ad space?

📊 Worked Business Example: A site wraps a thin slideshow in 12 ad spaces and sells impressions at $0.80 CPM, against $5 on a news page. The cheap price buys displays, but few of them reach a reader who came for the content.

⚖️ How to Judge the Number: There is no score with a zero point: MFA is a label given from site traits such as ad density, paid traffic and thin content. A CPM far below similar sites, such as $0.80 against $5, is a reason to check. Read the placement report against outcome measures, since MFA pages can show high viewability and few purchases.

⚠️ Common Misread to Avoid: Reading an MFA label as proof of bot traffic. The visitors can be real people sent there by paid links; fraud, viewability and suitability each need their own measure.

📐 Technical Definition: Verification vendors and industry bodies classify sites from layout and business practices. The simulator covers MFA in a note only; no number in it is an MFA rate or a share of wasted spend.

🎯 The Manager's Question: How many dollars did my dashboard hand to a channel that did not earn them?

📊 Worked Business Example: With only the email links untagged, the lab reports $49,995 misattributed: all of email's revenue, now sitting in Direct. With nothing tagged it is $125,110 of the month's $169,099.

⚖️ How to Judge the Number: $0 is the anchor, and a correct plan reaches it. Read it next to the budget you set: money credited to the wrong channel is money the CMO will move to the wrong channel. The total never changes, so compare the figure with the month's $169,099 to see how much of the picture was wrong.

⚠️ Common Misread to Avoid: Reading misattributed revenue as lost revenue. Nothing was lost; the same purchases happened and the total is identical in every column of the counterfactual picture. Only the credit moved.

📐 Technical Definition: Summed over converting sessions in the fixed 30-day log after the plan is applied. A source filed in the right group but split across several spellings is not misattributed here; the split-spelling note reports that separately.

🎯 The Manager's Question: After a "yes" period, how much higher are the odds of another "yes" the next period?

📊 Worked Business Example: Kestrel's outcome-lag odds ratio is 13.1 (interval 7.9 to 21.8): after an empty shelf, the odds of another are about 13 times higher. On a day with no truck and no promotion, that lifts the chance from about 10% to about 60%.

⚖️ How to Judge the Number: 1 means no momentum: yesterday says nothing beyond the drivers. Above 1 the outcome is sticky. Translate it at the no-driver rate: Kestrel's 13 lifts a 10% chance to 60%, while 5 would lift it only to 36%. Read it next to the driver odds ratios: momentum that survives with the drivers in the model is momentum they do not explain.

⚠️ Common Misread to Avoid: Reading momentum as a habit. In the Ferngrove Fresh case, sell-outs look sticky until rain is added: the odds ratio drops from 2.9 to 1.1, because rainy spells come in runs. A driver that comes in runs and is left out of the model shows up as momentum.

📐 Technical Definition: Last period's outcome enters as a 0/1 predictor, \(\ln\frac{P_t}{1-P_t} = \beta_0 + \phi\,y_{t-1} + \beta x_t\). The reading assumes the drivers that matter are in the model and the relationship held steady across the data. The tool's reference tests check the coefficients against statsmodels Logit to six decimals.

🎯 The Manager's Question: What range is plausible for this effect, given only the estimates and standard errors typed in?

📊 Worked Business Example: With the Artisan Bean case loaded (a = 0.44, SE 0.07; b = 0.58, SE 0.06), the tool draws 20,000 pairs of paths from those numbers. The middle 95% of their products runs from 0.17 to 0.35 around an indirect effect of 0.26.

⚖️ How to Judge the Number: Read it like any interval: if it excludes zero, the route is clear of no effect at the {level}% level. It can run slightly longer on the high side, because a product of two paths has a skewed spread. Read it next to the estimate, and when raw data exist, prefer the bootstrap interval.

⚠️ Common Misread to Avoid: A Monte Carlo interval is read as a bootstrap. Nothing was resampled, because the tool never saw the customers; it simulated paths from the numbers typed in. So it assumes the paths vary independently, which is close for a simple mediation and strained when paths share an equation.

📐 Technical Definition: Assumes each path estimate is roughly normal across samples. The tool makes 20,000 draws, forms the effect on each and reads the 2.5th and 97.5th percentiles. The draws are seeded, so every student sees the same interval for the same inputs.

🎯 The Manager's Question: Could chance placement alone produce this much grouping of like agents?

📊 Worked Business Example: At Year 21 of the default Brookhaven run, none of 199 random reshuffles matches the observed I of 0.57, so p = .005, the smallest value possible. The shuffled towns average I = 0.00 with SD 0.008, which puts the pseudo z near 70.

⚖️ How to Judge the Number: The usual line is {alpha}: below it, random placement is a poor explanation for the grouping. With 199 shuffles p cannot go below .005, so .005 means no shuffle matched the observed I. Read it next to Moran's I itself: on this 2,500-cell grid even an I of 0.05 clears the line, so the size of I decides whether the grouping matters.

⚠️ Common Misread to Avoid: Reading p = .005 as evidence for the households' rule. It says only that the layout is not random placement. Many different rules, or one shared outside influence, could leave the same clustering.

📐 Technical Definition: Group labels are shuffled across the analysed cells, with vacant lots left in place, and \(I\) is recomputed each time. The pseudo z is \((I_{obs} - \bar{I}^{(b)}) / \text{SD}(I^{(b)})\). The test runs only when the simulation is paused after Year 0, so playing at full speed is not slowed. The shuffles are seeded, so the same seed gives the same p.

🎯 The Manager's Question: How strongly do like agents sit next to like agents, compared with a random layout?

📊 Worked Business Example: Brookhaven starts at I = 0.00, a random scatter of households. When the default run settles at Year 21, I = 0.57: neighbors now share a group far more often than chance would give.

⚖️ How to Judge the Number: 0 means a random arrangement (strictly −1/(n−1), about −0.0005 on this grid). The card reads above 0.3 as clearly grouped, 0.1 to 0.3 as some grouping and below 0.1 as close to random. Negative values mean a checkerboard mix. Read it next to the permutation p-value, which says whether the grouping beats random placement, and next to the main outcome, which says what it adds up to.

⚠️ Common Misread to Avoid: Reading I as the share of agents living in clusters. An I of 0.57 does not mean 57% of households sit in one-group blocks. It is a correlation-like score: how strongly an agent's own group predicts its neighbors' groups, where 0 is random.

📐 Technical Definition: Group labels are categories, so the tool never codes them as 1, 2, 3. It computes \(I_c\) for a 0/1 marker of each group and reports \(I = \sum_c p_c I_c / \sum_c p_c\), weighted by group share \(p_c\). Neighbors are the 8 surrounding cells on a grid whose edges wrap, whatever the radius or network settings. Vacant lots are left out in the housing case. In 0/1 terms: at Year 21 a Red household's occupied neighbors are 72% Red on average, and a non-Red household's are 14% Red. That 58-point gap is close to I = 0.57.

🎯 The Manager's Question: If we add one more dollar to this channel next week, how many sales dollars come back?

📊 Worked Business Example: In the default Brightwell model the next dollar on TV returns $1.63 and the next dollar on search $1.24, though search's average ROI is twice TV's. That is why the reallocation step moves money from search into TV.

⚖️ How to Judge the Number: 1.00 is the sales break-even for the next dollar; profit needs about 1 divided by your gross margin. The tool marks a channel "Room to Grow" above 1.20 when that is at least 60% of its average ROI. Read it next to average ROI and across channels: money should move until the next dollar returns about the same everywhere.

⚠️ Common Misread to Avoid: Treating marginal ROI as fixed, so that doubling the budget doubles the return. It falls as spend rises: moving TV from $13,600 to $16,000 a week lowers its marginal ROI from $1.63 to $1.44. That fall, together with the shift limits, keeps the optimizer from putting everything into one channel.

📐 Technical Definition: This is the slope of the steady-state response curve \(Y_c(x) = \beta_c\, x / (x + K_c(1 - \lambda_c))\), the weekly contribution if spend held at x every week. The tool evaluates it at each channel's average weekly spend and checks it against a finite difference.

🎯 The Manager's Question: Do these items share enough common ground for a factor analysis to find anything?

📊 Worked Business Example: The default Environmental Consciousness items have an overall KMO of 0.84, in Kaiser's meritorious band. The sports item's own value (its MSA) is 0.40, the only one below 0.50, which singles it out before any factor is extracted.

⚖️ How to Judge the Number: It runs from 0 to 1; the tool warns below 0.60. Read the overall value next to each item's MSA in the table, where an item under 0.50 is a candidate to drop. Read it alongside Bartlett's test, which only checks that some correlation exists.

⚠️ Common Misread to Avoid: Treating KMO as a reliability score. A high KMO says the items share correlation that factors could explain. It says nothing about how many factors there are, or whether a summed score would be consistent; alpha and omega answer that.

📐 Technical Definition: Partial correlations come from the inverse of the correlation matrix. Large partials mean pairs of items are linked only to each other, not through a shared factor, which pulls KMO down. Each item's MSA is the same ratio over that item's row only. The tool matches factor_analyzer's calculate_kmo to five decimals.

🎯 The Manager's Question: Did the promotion go mostly to customers who would have bought anyway?

📊 Worked Business Example: Thistledown mailed coupons at random, and its coupon-average term is 0.81 with a standard error of 1.13: no sign of targeting. Fernhill's VIP coupon has an average term of 5.75, and with it in, the coupon's own coefficient falls from 1.10 to 0.46.

⚖️ How to Judge the Number: A value near 0, relative to its standard error, means the driver reached all kinds of customers alike; about 2 standard errors from 0 is the tool's significance flag. A large term means it went more to some kinds than others: targeting. Read it with the driver's own row; when targeting shows up, only that within-customer coefficient measures what the driver does.

⚠️ Common Misread to Avoid: Reading it as a second effect of the driver, to add to the first. It absorbs the difference between customers who tend to get the driver and those who do not. The coupon's own effect in Fernhill is 0.46, not 0.46 plus 5.75.

📐 Technical Definition: Mundlak (1978): entering \(\bar{x}_i\) beside \(x_{it}\) lets the baseline move with how much of the driver a customer usually gets. The average uses the fitted periods only, never the holdout. The tool flags a driver as targeted when its model C coefficient is under two-thirds of model A's (and A's exceeds 0.1 in size), or when its average term has p below 0.05.

🎯 The Manager's Question: How many cases is this answer built on?

📊 Worked Business Example: Aura's test rests on 1,850 clicks. Big Data Mode multiplies that by 100 at the same 3.8% rate, and the standard error falls only 10-fold, from 0.44 to 0.044 points.

⚖️ How to Judge the Number: No fixed number counts as enough; what you need depends on the smallest gap worth detecting. Read it next to the interval width: 4 times the cases halves it. With rare conversions, precision tracks the number of conversions (Aura's 70) more than the number of clicks.

⚠️ Common Misread to Avoid: Reading a big n as a safe answer. If the 1,850 clicks came only from Aura's most loyal followers, 18,500 more of the same would make the estimate more precisely wrong. Size fixes noise, not who was sampled.

📐 Technical Definition: The number of independent observations the test used, after missing values were dropped; the one-sample tool shows it as Active Sample (N). Repeated measures on the same person or store are not independent, and counting them separately overstates n.

🎯 The Manager's Question: What would a simple before-and-after or test-versus-control comparison have claimed?

📊 Worked Business Example: In the app-launch data, before-and-after says $45K and test-versus-control after launch says $50K. DiD says $30K, so the naive numbers overstate the lift by 50% to 67%.

⚖️ How to Judge the Number: Each naive number is the DiD estimate plus a bias: before-and-after adds the common trend ($15K here) and test-versus-control adds the baseline gap ($20K). Read them against the DiD estimate; the distance shows how far a simple report would have overclaimed. Neither belongs in a budget memo.

⚠️ Common Misread to Avoid: Taking whichever naive number is easiest to explain as "the effect". Before-and-after credits the {treatment} with market-wide drift, and test-versus-control credits it with the test markets being bigger to begin with. Only the difference of the two differences removes both.

📐 Technical Definition: Before-and-after equals \(\hat{\delta} + \Delta_C\), with \(\Delta_C\) the common trend. The post-period gap equals \(\hat{\delta} + \Delta_{\text{pre}}\), with \(\Delta_{\text{pre}}\) the baseline gap. So each is biased by exactly the term DiD removes.

🎯 The Manager's Question: Could the top few sets really be equally good?

📊 Worked Business Example: Frost & Fold's best set reaches 85.3% of 320 shoppers, with a margin of 3.9 points. Nine other sets reach 82.5% or more, so all ten belong on the shortlist.

⚖️ How to Judge the Number: Sets within the margin of the leader cannot be told apart with this sample. The tool uses the larger of the 95% margin on the leader's reach and 1 percentage point. Read it next to cost, supply and brand fit, which decide between tied sets.

⚠️ Common Misread to Avoid: Reading the first-listed set as the winner. Inside the margin the ordering is noise, and a fresh sample of 320 would often reorder the top ten. Choose among them on things the survey did not measure.

📐 Technical Definition: The margin is the half-width of the usual 95% interval on a proportion. Kish's effective n is \((\sum w_i)^2 / \sum w_i^2\), which shrinks as the weights grow more unequal.

🎯 The Manager's Question: Of the customers the model passes over, what share really were no customers?

📊 Worked Business Example: At the 0.50 cutoff in the influencer preset the model passes over 639 posts, and 453 of them made no sale: an NPV of 70.9%. The other 186 are converting posts the model missed.

⚖️ How to Judge the Number: Passing customers over at random gives an NPV equal to the share of no answers, 53.7% here, so that is the floor to beat. Read it together with specificity at the same cutoff. It falls as the outcome becomes more common, because more yeses hide among the passed-over.

⚠️ Common Misread to Avoid: Reading NPV as specificity. Specificity divides by all actual no customers (453 of 537, 84.4%); NPV divides by everyone the model passed over (453 of 639, 70.9%). They answer different questions about the same table.

📐 Technical Definition: It cannot be computed when the model flags everyone. It is the precision of the "no" calls, and it rises as the cutoff is lowered: 77.9% at 0.30 against 65.6% at 0.70 in the influencer preset.

🎯 The Manager's Question: How many nearby agents does each agent watch when it decides?

📊 Worked Business Example: Radius 1 means the 8 surrounding cells, 2 means 24 and 3 means 48. Brookhaven's default run settles at 71.7% same-group with r = 1 and at 74.1% with r = 2.

⚖️ How to Judge the Number: r = 1 is the default and the smallest. Each step adds a ring of cells, and the edges wrap so every agent has the same count. A wider view averages over more neighbors, so one defector moves an agent's share less, but news travels further. Read it next to Moran's I, which the tool always measures on the 8 nearest cells whatever the radius.

⚠️ Common Misread to Avoid: Assuming a wider circle always calms a market down. A wider view smooths out small local pockets, but it also carries a large pocket's influence further. In Brookhaven, r = 2 sorts the town slightly more than r = 1, not less.

📐 Technical Definition: Neighbors are all cells within Chebyshev distance r on a grid whose edges wrap (a torus). With small-world topology on, each link is rewired to a random distant agent with chance 0.20.

🎯 The Manager's Question: What share of the customer base is still active, either loyal or delighted?

📊 Worked Business Example: StreamFlow starts at 88.0% active and, with no outage, climbs to 99.2% by Year 30. After an outage at Year 10 it drops to 71.0%, and by Year 60 it is back only to 86.3% against 99.0% with no outage.

⚖️ How to Judge the Number: 100% means every seat holds a loyal or delighted customer; the default starts at 88.0%. Read it next to the dashed no-shock line, whose distance from this one is the shock's effect, and next to Churned Customers. Use replications before quoting that distance, since one run is one draw.

⚠️ Common Misread to Avoid: Treating a large delighted group as protection. Delighted customers tolerate 15 more points of complaints than loyal ones, so they sour later, not never. After the outage the delighted share falls from 17.4% to 13.0% by Year 60.

📐 Technical Definition: Dissatisfied and churned seats count as inactive. The run settles when nobody changes state for 5 years, or when the share stays inside a 1-point band for 10 years.

🎯 The Manager's Question: How does each unit turn the values coming in into the number it passes on?

📊 Worked Business Example: A unit gets standardized tenure 0.5 and emails −1.2, with weights 0.8 and 0.3 and a bias of 1.0. Its sum is 0.8 × 0.5 + 0.3 × (−1.2) + 1.0 = 1.04, and tanh squashes that to 0.78 for the next layer.

⚖️ How to Judge the Number: Multiply each incoming value by its weight, add the bias, then apply the activation. The anchor is a sum of 0, which gives tanh 0, the middle of its −1 to 1 range, and leaves ReLU switched off. Read each weight together with the value it multiplies, because a big weight on a near-0 value adds almost nothing.

⚠️ Common Misread to Avoid: A large weight is not the input’s overall importance or a causal effect. Its contribution depends on the incoming value, the activation and every later layer. Permutation importance answers the overall question.

📐 Technical Definition: Hidden layers apply tanh or ReLU. A yes/no output applies the sigmoid, category outputs apply softmax across their scores together, and a number output stays linear and is converted back from the training scale. Every unit in a layer reads every output of the layer before.

🎯 The Manager's Question: How strongly does the customer review support the target construct hypothesis?

📊 Worked Business Example: In Canyon Creek Bistro, a review saying "we waited 45 minutes past our reservation" scores 0.94 on Service Delay. A review praising fast seating scores 0.04.

⚖️ How to Judge the Number: Scores run from 0.00 to 1.00, with 0.50 as the default presence cutoff read alongside construct prevalence and key-driver slopes.

⚠️ Common Misread to Avoid: Reading 0.90 as 90% of reviewers mentioning the topic. It is the model probability for one single document, not a sample share.

📐 Technical Definition: In multi-label mode, each construct is scored on its own. The model drops the neutral logit and applies a two-way softmax over entailment and contradiction.

🎯 The Manager's Question: How does the sentence frame around a construct label shape what the NLI model scores?

📊 Worked Business Example: Framing "Price Fairness" inside "This review complains about {label}" scores a 1-star review at 0.91. Framing it inside "This review praises {label}" scores it at 0.03.

⚖️ How to Judge the Number: Compare mean construct probabilities on the 0.00 to 1.00 scale across templates alongside the key-driver regression slopes.

⚠️ Common Misread to Avoid: Assuming the template wording is cosmetic filler. Changing the verb from neutral "discusses" to directional "complains about" shifts scores from topic detection to defect detection.

📐 Technical Definition: Transformer NLI encoders attend jointly to the review premise and the full hypothesis string H_k. Verbs like "complains" or "praises" condition attention on valence cues.

🎯 The Manager's Question: Can a single review score high on several constructs at once, or must probabilities sum to one?

📊 Worked Business Example: A review complaining about both a 40-minute wait and cold halibut scores 0.93 and 0.89 in Multi-Label mode. Single-Label mode forces those two scores down near 0.48 each so the row sums to 1.00.

⚖️ How to Judge the Number: Use Multi-Label (0.00 to 1.00 per construct) for regression, and read Single-Label (row sum = 1.00) alongside VIF diagnostics.

⚠️ Common Misread to Avoid: Using Single-Label mode when reviews discuss multiple topics. Forcing row probabilities to sum to 1.00 penalizes multi-issue reviews and inflates regression VIF.

📐 Technical Definition: Multi-label mode applies a binary softmax over entailment and contradiction logits for each construct. That avoids the unit-simplex constraint.

🎯 The Manager's Question: How strongly does this document or keyword belong to an extracted latent theme?

📊 Worked Business Example: Factorizing 150 hotel reviews into 4 themes gives "pool" and "cabana" loadings above 0.45 on Theme 2. Generic words load near 0.00.

⚖️ How to Judge the Number: Loadings are non-negative (0.00 or higher). Compare a loading against its loadings on the other themes.

⚠️ Common Misread to Avoid: Treating NMF themes as mutually exclusive buckets where a review can only touch one topic. Each document receives a continuous vector of loadings across all extracted themes.

📐 Technical Definition: Both W and H are constrained to be non-negative. That forces NMF to learn an additive parts-based representation without negative cancellations.

🎯 The Manager's Question: Do the model's misses follow a bell curve, or do big misses happen more often than one allows?

📊 Worked Business Example: In the 60 ad campaigns, the lowest points fall well below the line: the three smallest budgets missed by $15k to $22k on the low side. That sag at the bottom is the long left tail behind a skewness of -0.86.

⚖️ How to Judge the Number: Points should hug the diagonal from end to end. A few points peeling away at one end are single extreme cases; an S shape means the tails are heavier or lighter than a bell curve. Read it together with the Shapiro-Wilk p-value (0.017 here) and the sample size: with 30 or more rows, mild bends matter little.

⚠️ Common Misread to Avoid: Checking the raw outcome instead of the residuals. Sales revenue can be skewed while the misses are bell-shaped, because the predictor accounts for the skew. The plot is about the misses only.

📐 Technical Definition: Each ordered standardized residual \(r_{(i)}\) is plotted against \(q_i\), where \(r_{(i)}\) is the ith smallest standardized residual. The fractions \((i-0.375)/(n+0.25)\) are Blom's plotting positions.

🎯 The Manager's Question: Why does this report have a row with no name, and does it matter?

📊 Worked Business Example: The lab's Campaigns report shows summer-glow with $125,110 beside (not set) with $32,151. The second row is every visit that carried no utm_campaign: all organic and type-in traffic, plus any paid link left untagged.

⚖️ How to Judge the Number: In a campaign or content report, (not set) should hold only traffic you never sent: organic search, type-in, press, about $32,151 here. Read it next to your tagging plan: if it grows when a paid source is untagged, that source is hiding inside it. Zero is not the goal; knowing what is in there is.

⚠️ Common Misread to Avoid: Reading (not set) as a bug in the analytics tool. It is the honest answer when a dimension has no value for that visit. The fault, if there is one, is the untagged link that put a paid visit in it.

📐 Technical Definition: The lab writes (not set) whenever a session's utm_campaign or utm_content is empty after the plan is applied. Rows named (not set) are left out of the comparison picture and kept in the full table.

🎯 The Manager's Question: Has this test shown a gap I can act on, or is it still too close to call?

📊 Worked Business Example: Had Aura stopped at 300 clicks with 11 sales (3.7%), p against the 2.4% baseline would be 0.08: not significant at 0.05. Yet its interval, 1.5% to 5.8%, still holds lifts worth tens of thousands of dollars a year.

⚖️ How to Judge the Number: This data cannot separate the gap from ordinary variation at the {alpha} line. That is a statement about what the sample can resolve, not evidence that the gap is zero. Read it next to the confidence interval: if the interval still holds differences worth acting on, collect more data rather than drop the idea.

⚠️ Common Misread to Avoid: Reading "not significant" as "no difference". Aura's 300-click run misses the line, yet its interval runs up to 5.8%, more than double the baseline. The test failed to find a gap, which is not the same as finding that there is none.

📐 Technical Definition: Failing to reject \(H_0\) is not the mirror image of rejecting it. The test caps the false-alarm rate at {alpha} but says nothing about the miss rate, which depends on power and so on sample size. A 300-click test has little power against a lift of 1.4 points.

🎯 The Manager's Question: When the model does not flag a customer, how safe is it to leave them alone?

📊 Worked Business Example: In the churn preset the model clears 884 subscribers, and 882 of them really stay. NPV is about 99.8%: only 2 churners slip through unflagged.

⚖️ How to Judge the Number: The anchor is 1 minus the base rate: a model that never flags anyone already scores 98% here when churn is 2%. So read it against that floor, together with sensitivity, before trusting the cleared list. A value close to the floor means the model adds little for the customers it passes over.

⚠️ Common Misread to Avoid: Reading a 99.8% NPV as proof the model is excellent. When only 2% churn, flagging nobody scores 98% and catches no one. The gain over that 98% floor, and the sensitivity, show what the model is worth.

📐 Technical Definition: By Bayes' rule it equals \(P(\neg B \mid \neg A)P(\neg A) / [P(\neg B \mid \neg A)P(\neg A) + P(\neg B \mid A)P(A)]\): specificity weighted by the share without the event. The lab computes it exactly from its sliders. It cannot be computed when the model flags everyone.

🎯 The Manager's Question: What share of the emotionally loaded words in this text evoke a specific discrete emotion?

📊 Worked Business Example: A review matches 8 NRC lexicon words, and 4 of them carry Fear associations. Its Fear emotion share is 50%.

⚖️ How to Judge the Number: Shares run from 0.00 to 1.00 across Plutchik’s 8 emotions, read alongside lexicon coverage and VADER valence.

⚠️ Common Misread to Avoid: Assuming two negative reviews with the same VADER score need the same response. High Anger demands service recovery whereas high Fear or Disgust signals a safety or quality risk.

📐 Technical Definition: One English word can evoke multiple Plutchik emotions in Mohammad and Turney (2013). As a result, the eight emotion shares need not sum to 1.00.

🎯 The Manager's Question: How often would a gap at least this big show up if nothing real were going on?

📊 Worked Business Example: Aura Athletics converts 70 of 1,850 ad clicks (3.8%) against its 2.4% baseline. The p-value is about 0.00005: if the true rate were still 2.4%, a sample this high would turn up about 5 times in 100,000.

⚖️ How to Judge the Number: Smaller means this data would be more surprising if there were no real difference. The usual line is {alpha}. Below it, act as if the difference is real; above it, you have not shown one, which is not the same as showing there is none. Read it next to the effect size and the sample size, because a huge sample makes a trivial gap "significant".

⚠️ Common Misread to Avoid: A p-value of 0.03 is not a 3% chance that the result is a fluke. Nor is it a 97% chance that the effect is real. It answers the opposite question: if there were no difference at all, data this far out would appear 3% of the time.

📐 Technical Definition: One-sided tests use the one tail the hypothesis names; two-sided tests double it. The value is only as good as the null model behind it, so broken assumptions (dependence, the wrong spread) give a misleading p.

🎯 The Manager's Question: How well could we predict the outcome knowing nothing about the customers?

📊 Worked Business Example: In the influencer data 463 of 1,000 posts converted, so the null model gives every post a 46.3% chance. Its log-likelihood, -690.4, is the starting point the predictors must improve on.

⚖️ How to Judge the Number: Every row gets the same probability: the overall rate, here 46.3%. The model chi-square compares the fitted model against this baseline on the same rows. A significant improvement shows the predictors carry information, not that the predictions are good on new rows.

⚠️ Common Misread to Avoid: Thinking the null model predicts the most common outcome for everyone. It gives everyone the overall rate, 46.3%, not 0%. Always choosing "no sale" is a different baseline, for accuracy, and it is right 53.7% of the time.

📐 Technical Definition: Its intercept is \(\hat{\beta}_0 = \ln[\hat{p}_0/(1-\hat{p}_0)]\), with no slopes. The likelihood-ratio test compares it with the predictor model fitted on the same rows. Both outcomes must appear in the data for the intercept to be finite.

🎯 The Manager's Question: How many customers actually landed in this category in our sample?

📊 Worked Business Example: Of Zenith's 1,250 taste testers, 380 picked Berry Blast and 130 picked Matcha Zen. Those are observed counts, and the five of them add up to 1,250.

⚖️ How to Judge the Number: On its own a count has no benchmark, so compare it with the expected count E in the same row. Berry Blast's 380 against an expected 250 is 130 above; Matcha Zen's 130 is 120 below. Whether that gap matters is what the chi-square component and the p-value decide.

⚠️ Common Misread to Avoid: Judging a category by its raw count. A count of 380 looks big, but it only means something against what the benchmark expects: 250 under an equal split, or 475 if the benchmark gave Berry Blast 38%. Compare O with E, never O alone.

📐 Technical Definition: Counts must be whole numbers of independent cases, one category per case. The test uses counts, not percentages: the same shares from 125 testers would give a chi-square one tenth as large. The tool's big-data switch multiplies every count to show this.

🎯 The Manager's Question: How many customers actually landed in this category in our sample?

📊 Worked Business Example: Of Zenith's 1,250 taste testers, 380 picked Berry Blast and 130 picked Matcha Zen. Those are observed counts, and the five of them add up to 1,250.

⚖️ How to Judge the Number: On its own a count has no benchmark, so compare it with the expected count E in the same row. Berry Blast's 380 against an expected 250 is 130 above; Matcha Zen's 130 is 120 below. Whether that gap matters is what the chi-square component and the p-value decide.

⚠️ Common Misread to Avoid: Judging a category by its raw count. A count of 380 looks big, but it only means something against what the benchmark expects: 250 under an equal split, or 475 if the benchmark gave Berry Blast 38%. Compare O with E, never O alone.

📐 Technical Definition: Counts must be whole numbers of independent cases, one category per case. The test uses counts, not percentages: the same shares from 125 testers would give a chi-square one tenth as large. The tool's big-data switch multiplies every count to show this.

🎯 The Manager's Question: What are the smallest and largest values the model has actually seen?

📊 Worked Business Example: In the influencer preset story views run from 200 to 120,781 per post. A what-if profile at 200,000 views asks the model about a post unlike any it was fitted on.

⚖️ How to Judge the Number: They are the edges of the data in its own units, and predictions far outside them are extrapolation. Check an extreme against the median first: a maximum 16 times the median of 7,399 views is real but rare. Read them alongside the SD, and keep simulator values inside the range.

⚠️ Common Misread to Avoid: Reading the minimum and maximum as limits on future customers. They are just the two most extreme rows in this file, so one unusual post can move either end. They are also not a confidence interval for anything.

📐 Technical Definition: The smallest and largest usable values in the rows the model was fitted on. Neither estimates a population boundary.

🎯 The Manager's Question: How many customers say yes for every customer who says no?

📊 Worked Business Example: In the influencer preset 463 of 1,000 posts converted, so the odds are 463 to 537, or 0.86. A post the model gives an 85% chance has odds of 0.85 to 0.15, about 5.7.

⚖️ How to Judge the Number: Odds of 1 mean an even 50% chance; 0.25 is a 20% chance and 4 is 80%. Odds run from 0 upward with no ceiling, so they stretch out fast above 50%. Read them next to the probability, which is what a contact plan or a forecast needs.

⚠️ Common Misread to Avoid: Reading odds of 3 as a 300% or a 30% chance. Odds of 3 mean three yeses for every no, a 75% chance.

📐 Technical Definition: Going back, \(p = \text{odds}/(1 + \text{odds})\). In logistic regression the fitted odds are \(e^{\eta}\), so each coefficient multiplies them by \(e^{\beta}\), the odds ratio.

🎯 The Manager's Question: How much does this predictor multiply the odds of a "yes", with the others held fixed?

📊 Worked Business Example: An odds ratio of 1.8 for "saw the influencer post" means viewers have 1.8 times the odds of converting. If non-viewers convert at 5% (odds of 0.053), viewers convert at about 8.7%, not 9%.

⚖️ How to Judge the Number: 1 means no association. 2 doubles the odds, 0.5 halves them, and 0.5 and 2 are the same strength in opposite directions. If the interval includes 1, this data cannot tell the predictor apart from no effect. Read it next to the percentage-point effect, which is what a budget decision needs.

⚠️ Common Misread to Avoid: Reading "odds ratio 2" as "twice as likely". It doubles the odds, not the probability: doubling the odds of a 5% event takes it to about 10%, but doubling the odds of a 60% event takes it only to 75%.

📐 Technical Definition: The interval is \(e^{\beta \pm z_{1-\alpha/2}\,\text{SE}(\beta)}\): the log-odds limits exponentiated, which is why it is not symmetric around the estimate. For a continuous predictor the ratio is per one-unit step in its own units.

🎯 The Manager's Question: Do these items hang together once strong and weak items are allowed to count differently?

📊 Worked Business Example: On the default Environmental Consciousness items, omega is 0.72 while alpha is 0.69. Omega weights each item by its loading, so the near-zero sports item drags it down less.

⚖️ How to Judge the Number: Read it on the same bands as alpha, with 0.70 as the usual line. Read it next to alpha: a gap of a few hundredths is normal, and a gap of 0.05 or more says the items are unequal, so report omega. The whole-scale omega assumes one factor, so check that parallel analysis keeps one first.

⚠️ Common Misread to Avoid: Treating omega and alpha as rival verdicts and quoting the higher one. Omega is alpha without the equal-weights assumption, so when they differ, omega is the better estimate of the same thing. On the default data they agree: 0.72 and 0.69 both sit near the 0.70 line.

📐 Technical Definition: The tool fits one factor by principal axis factoring and puts its standardized loadings into the formula, so omega assumes a single factor and uncorrelated errors. For a scale with several factors, use the per-factor omega in the validity table instead. On one factor's items it is the same number as composite reliability.

🎯 The Manager's Question: What share of outcome variation in the broader population is explained by the groups?

📊 Worked Business Example: A 4-group study might show η² = 0.09 in the sample. After subtracting expected noise, ω² drops to 0.06.

⚖️ How to Judge the Number: Compare against 0.01 (small), 0.06 (medium), and 0.14 (large) alongside η² and the omnibus F-test.

⚠️ Common Misread to Avoid: Panicking when ω² is smaller than η² or clamped at zero. Zero ω² means between-group spread was no larger than random noise.

📐 Technical Definition: Subtracting \(df_{\text{between}} MS_{\text{within}}\) in the numerator removes the positive sample bias of \(\eta^2\) under the null hypothesis.

🎯 The Manager's Question: What share of outcome variation in the broader population is explained by the groups?

📊 Worked Business Example: A 4-group study might show η² = 0.09 in the sample. After subtracting expected noise, ω² drops to 0.06.

⚖️ How to Judge the Number: Compare against 0.01 (small), 0.06 (medium), and 0.14 (large) alongside η² and the omnibus F-test.

⚠️ Common Misread to Avoid: Panicking when ω² is smaller than η² or clamped at zero. Zero ω² means between-group spread was no larger than random noise.

📐 Technical Definition: Subtracting \(df_{\text{between}} MS_{\text{within}}\) in the numerator removes the positive sample bias of \(\eta^2\) under the null hypothesis.

🎯 The Manager's Question: How many yes and no customers does the model have to learn from?

📊 Worked Business Example: The influencer preset has 463 posts that converted and 537 that did not, 1,000 in all. With 7 predictor terms that is about 66 of the rarer answer per term, plenty for a stable fit.

⚖️ How to Judge the Number: What limits the model is the rarer answer, not the total. A common rule of thumb asks for at least 10 of the rarer answer per predictor term; below that, coefficients and intervals turn unstable. Read it next to the base rate and the number of terms in the coefficient table.

⚠️ Common Misread to Avoid: Reading a large total as enough data. A file of 10,000 customers with 40 buyers gives the model only 40 examples of buying. That is 4 per term for a 10-term model, too few for stable coefficients.

📐 Technical Definition: The number of usable rows in each outcome category; the two counts sum to the n used for fitting. The 10-per-term guide comes from Peduzzi and colleagues (1996).

🎯 The Manager's Question: Is this customer unlike any the models learned from, so the prediction is a guess?

📊 Worked Business Example: Wrenfield’s markets never spent more than $25,000 on social. Set every held-back market to $60,000 and the partial dependence chart averages $304,000 of revenue for the regression, $254,000 for the ReLU network and $220,000 for tanh, where "Reveal the truth" also sits.

⚖️ How to Judge the Number: Any flag means treat that prediction as a guess, so 0 flagged rows is the target. The row is still scored, because a model can always produce a number. Read the flag together with the partial dependence chart, whose shaded stretch shows what each model does past the data.

⚠️ Common Misread to Avoid: A network does not handle new territory better than a straight line. Each model continues its own built-in shape: a regression keeps its slope, tanh flattens and ReLU runs on straight. At Wrenfield’s $60,000 the three predictions span $84,000, and which one lands close is luck.

📐 Technical Definition: A scored row is flagged when any numeric input is below its training minimum or above its training maximum; a value exactly at a limit is inside. A category the training rows never had is refused instead (unseen level). The check runs one input at a time, so an unusual mix of in-range values is not flagged.

🎯 The Manager's Question: Do the predictors, taken together, predict {outcome} better than just using its average?

📊 Worked Business Example: For the 60 ad campaigns F is about 378 on 1 and 58 degrees of freedom, with p below 0.001. Spend clearly beats the average, which says nothing yet about whether a straight line is the right shape.

⚖️ How to Judge the Number: If the predictors explained nothing, F would sit near 1. A p-value below {alpha} says they beat the average together; it does not say which one matters. Read it next to the individual coefficient tests and the diagnostics.

⚠️ Common Misread to Avoid: Reading a significant F as "the model is good". It only rules out the idea that none of the predictors matter. The ad campaigns pass it easily and still fail the linearity check.

📐 Technical Definition: Here \(SS_{\text{res}} = \sum e_i^2\), where \(e_i\) is the residual for case i. F is compared with an F distribution on \(k-1\) and \(n-k\) degrees of freedom, and assumes normal errors with equal spread.

🎯 The Manager's Question: How much extra customer-to-customer variance sits above the Poisson baseline?

📊 Worked Business Example: In CraveDrop, estimated alpha is 0.67, so a segment averaging 4.0 orders has variance 4.0 + 0.67(16.0) = 14.7 instead of 4.0. Accounting for that spread widens standard errors and prevents false alarms.

⚖️ How to Judge the Number: 0.00 reduces Negative Binomial to standard Poisson. Values above 0.10 indicate substantial extra variance, read alongside the overdispersion likelihood-ratio test and Pearson dispersion.

⚠️ Common Misread to Avoid: Treating alpha as a predictor slope that changes the average count. Alpha leaves expected counts nearly unchanged and instead adjusts the variance law and standard errors for unobserved customer heterogeneity.

📐 Technical Definition: NB2 arises when individual Poisson rates include an unobserved Gamma frailty term with variance \(\alpha\). Maximum likelihood estimates \(\alpha\) jointly with the slopes.

🎯 The Manager's Question: Does the model forecast a customer's next weeks better than that customer's own buying history?

📊 Worked Business Example: On Thistledown's 8 held-back weeks, model C scores a log loss of 0.356 against 0.366 for each shopper's own rate. The overall average rate scores 0.477, so beating it says little.

⚖️ How to Judge the Number: Lower log loss is better, and the bar is the own-rate number, not the overall rate. Beating it by 0.01, as model C does in Thistledown, is a real but small gain. Read it next to the AUC in the same table; a model that loses to own rate should not drive targeting.

⚠️ Common Misread to Avoid: Celebrating a model because it beats the overall average rate. When customers differ this much, any forecast that knows who is who wins easily. Model A beats the overall rate in Thistledown (0.430 against 0.477) and still loses to own rate (0.366).

📐 Technical Definition: It is updated as each held-back period passes, so it always uses the latest history. The 0.5 and the 1 keep a customer with no purchases off a forecast of exactly zero, which log loss would punish without limit. It is scored on the same held-back periods as the models.

🎯 The Manager's Question: How often would a gap at least this big show up if nothing real were going on?

📊 Worked Business Example: Aura Athletics converts 70 of 1,850 ad clicks (3.8%) against its 2.4% baseline. The p-value is about 0.00005: if the true rate were still 2.4%, a sample this high would turn up about 5 times in 100,000.

⚖️ How to Judge the Number: Smaller means this data would be more surprising if there were no real difference. The usual line is {alpha}. Below it, act as if the difference is real; above it, you have not shown one, which is not the same as showing there is none. Read it next to the effect size and the sample size, because a huge sample makes a trivial gap "significant".

⚠️ Common Misread to Avoid: A p-value of 0.03 is not a 3% chance that the result is a fluke. Nor is it a 97% chance that the effect is real. It answers the opposite question: if there were no difference at all, data this far out would appear 3% of the time.

📐 Technical Definition: One-sided tests use the one tail the hypothesis names; two-sided tests double it. The value is only as good as the null model behind it, so broken assumptions (dependence, the wrong spread) give a misleading p.

🎯 The Manager's Question: On customers neither model saw, does the network beat the regression by more than luck?

📊 Worked Business Example: On Ridgeline Roasters’ 800 held-back subscribers, the regression’s log loss is 0.561 and the network’s 0.519, a gap of 0.042. The 95% interval runs from 0.026 to 0.060, entirely above 0, so the page calls the network better.

⚖️ How to Judge the Number: It is the regression’s holdout score minus the network’s, so 0 is a tie and a positive gap favors the network. Read the interval, not the single number. Entirely above 0, the network is better; including 0, there is no real difference, so keep the regression you can explain; entirely below 0, the regression is better. Read it next to the no-model guess: at Ridgeline that guess scores 0.604, so the regression beats it by 0.043 and the network by twice that.

⚠️ Common Misread to Avoid: Any positive gap is not a win. If the interval includes 0, a different draw of held-back customers could just as easily have favored the regression. Copperline’s gap of −0.001, with an interval from −0.007 to 0.005, is a tie.

📐 Technical Definition: The interval comes from 1,000 resamples of the holdout rows, drawing the same rows for both models each time. On each resample the gap is recomputed, and the interval is the 2.5th to 97.5th percentile of those gaps. Pairing removes how hard each customer is to predict for both models, which is what makes the interval narrow enough to be useful.

🎯 The Manager's Question: How many distinct ideas do these items measure, beyond what random answers would produce?

📊 Worked Business Example: In the default survey the first eigenvalue is 2.85 against a random-data benchmark of 1.54, and the second is 1.05 against 1.34. Parallel analysis keeps one factor; Kaiser's eigenvalue-above-1 rule would keep two.

⚖️ How to Judge the Number: Keep a factor only while its eigenvalue beats the 95th percentile of eigenvalues from random data of the same size, 500 simulated datasets here. The nothing-there benchmark is random data, not 1. Read the count next to the scree plot and the loadings: a factor with fewer than 3 clear items is too thin to name.

⚠️ Common Misread to Avoid: Trusting Kaiser's rule and keeping every factor with an eigenvalue above 1. In the default data that keeps a second factor at 1.05, though random answers alone reach 1.34 in that spot. Parallel analysis drops it, and the lab uses its count unless you override it.

📐 Technical Definition: The tool simulates 500 datasets of independent normal noise with the same n and p, seeded so a class sees the same benchmark, and takes the 95th percentile at each position. Eigenvalues come from the full correlation matrix. It counts every position whose real eigenvalue clears its benchmark, which in practice equals stopping at the first that does not.

🎯 The Manager's Question: How many numbers does the network tune for each customer it learns from?

📊 Worked Business Example: Ridgeline’s default network has 241 weights and biases on 2,399 training rows, about 0.10 per row. Tallyhop’s preset of two 32-unit layers has 1,793 on 239 trials, about 7.5 per row, and memorizes them.

⚖️ How to Judge the Number: 1 is the line: above it the network has more dials than rows and can memorize every row, so the page raises a caution. Below 1 there is no safe cutoff, so read it next to the gap between training and validation loss. For scale, a logistic regression on Tallyhop’s 21 inputs has 22 parameters, about 0.09 per row.

⚠️ Common Misread to Avoid: A small ratio does not mean the network cannot overfit. Training for too long can still memorize noise, so the validation curve, not the ratio, says when memorizing has started.

📐 Technical Definition: Each unit has one weight per incoming value plus one bias, which is the "+ 1". A category with k levels enters as k − 1 inputs. The output layer has one unit for a yes/no or a number and one per level for a category.

🎯 The Manager's Question: As one input moves, how does the average prediction move, and in what shape?

📊 Worked Business Example: At Ridgeline the network’s curve for emails_per_month is a U: predicted churn is lowest near 4.6 emails a month and rises toward both 0 and 12. The regression, with one straight-line effect per input, cannot draw that U.

⚖️ How to Judge the Number: Read the shape. A flat line at the overall rate (29% churn at Ridgeline) means the input barely moves the prediction. A steady rise or fall is a straight push, and a U or a bend is a curve a plain regression misses. Compare the network’s line against the baseline’s on the same chart. A shaded stretch lies outside the training data, where neither model has evidence.

⚠️ Common Misread to Avoid: Not every customer follows this curve. It is an average over customers: at Ridgeline the save offer looks modest on average, yet it cuts churn sharply for new subscribers and barely moves established ones. It is also an association, not what changing the input would do.

📐 Technical Definition: Computed over the holdout rows on a 25-point grid from the training 5th to 95th percentile. "Show beyond the data" extends it to the training maximum plus 1.5 times the training range. Correlated inputs blur it, because setting one input for everyone builds customers who may not exist.

🎯 The Manager's Question: How much does the downstream variable shift when the upstream one rises by one unit?

📊 Worked Business Example: In the Artisan Bean raw data, the b path is 2.22. Customers who rate quality one point higher on the 7-point scale are willing to pay $2.22 more on average at the same price tier.

⚖️ How to Judge the Number: Zero means no link once the other arrows into the same box are held fixed. Its size is in the downstream variable's units per unit of the upstream one: 2.22 dollars per quality point, or 0.58 standard deviations in the standardized case. Read it next to its standard error and interval, and against what a one-unit change costs to achieve.

⚠️ Common Misread to Avoid: A path coefficient is read as the correlation between the two boxes. It is a partial slope: the b path holds {predictor} fixed, so it can be small even when the mediator and {outcome} are strongly correlated.

📐 Technical Definition: Assumes each equation is linear and correctly specified. The tool fits one equation per box with arrows into it, by ordinary least squares, and reports slopes in the units of your columns. A binary outcome is fitted by logistic regression, so its arrows are on the log-odds scale. The case studies are in standard-deviation units.

🎯 The Manager's Question: Which factor does each item belong to, once the overlap between factors is taken out?

📊 Worked Business Example: In PureGlow, "Truthful ingredient labeling" has a pattern loading of 0.80 on Authenticity and near 0 on the others, so it belongs to Authenticity alone. Its structure correlation with Efficacy is still 0.29, because those two factors correlate 0.28.

⚖️ How to Judge the Number: Read each row for one big loading and the rest near 0; the big one names the item's factor, and 0.40 is the tool's line for a clear loading. Read it alongside the factor correlations. When factors correlate 0.30 or more, pattern and structure loadings differ, and the pattern matrix is the one for assigning items.

⚠️ Common Misread to Avoid: Reading a pattern loading as the item's correlation with the factor. After Promax it is a regression weight with the other factors held fixed, the item's unique link. The plain correlations are in the structure matrix, and they run larger whenever factors correlate.

📐 Technical Definition: After an oblique rotation the factors correlate, \(\Phi = E[\mathbf{f}\mathbf{f}^{\top}] \ne I\), so each item needs two matrices: the pattern \(\Lambda\) and the structure \(S = \Lambda \Phi\). The tool runs Promax (power 4) after principal axis factoring, then sorts the factors and flips signs so the largest loadings are positive.

🎯 The Manager's Question: How strongly does this survey item reflect the idea the factor stands for?

📊 Worked Business Example: In the default one-factor solution, "Support green policies" loads 0.65 and "Watch sports on weekend" loads 0.03. The first is a good marker of environmental concern; the second measures something else entirely.

⚖️ How to Judge the Number: Loadings run from -1 to 1, and 0 means no link to the factor. The tool calls a primary loading below 0.40 weak, and such an item is a candidate to drop or rewrite; the chapter calls 0.70 and up strong. Read it next to the item's communality and its second-highest loading, since an item can load 0.50 and still cross-load.

⚠️ Common Misread to Avoid: Reading a loading of 0.60 as "the item explains 60% of the factor". On a single factor a loading is the item's correlation with the factor, so its square is the share of the item the factor explains. Here that is 0.60 squared, 0.36, about a third.

📐 Technical Definition: The tool extracts factors by principal axis factoring, with principal components as the alternative. It then rotates with Promax, which lets factors correlate. The loadings shown are the pattern matrix: each is a regression weight with the other factors held fixed, not a plain correlation.

🎯 The Manager's Question: How closely do {predictor} and {outcome} move together along a straight line?

📊 Worked Business Example: Across Brightline Retail's 48 weekly campaigns, ad spend and revenue correlate at r = 0.68. Weeks with higher spend tend to bring in higher revenue.

⚖️ How to Judge the Number: It runs from -1 (opposite directions) through 0 (no straight-line pattern) to +1 (lockstep). Read it next to R-squared (r squared) for shared variation and the scatterplot for shape or outliers.

⚠️ Common Misread to Avoid: Reading r = 0.68 as proof that raising {predictor} causes {outcome} to rise. Correlation is symmetric and blind to confounders: holiday demand could lift both spend and revenue at once.

📐 Technical Definition: Because each deviation is squared in the denominator, one extreme case far from the mean can dominate r. Confidence intervals use Fisher's z transform, \(z = \tfrac{1}{2}\ln\frac{1+r}{1-r}\), with standard error \(1/\sqrt{n-3}\).

🎯 The Manager's Question: How many times larger is the observed count spread than a Poisson model expects?

📊 Worked Business Example: In Thistledown Bakery, Pearson dispersion is 1.02, matching the Poisson assumption. In CraveDrop, Pearson dispersion under Poisson reaches 3.41, warning that actual variance is more than three times the mean.

⚖️ How to Judge the Number: 1.00 marks exact Poisson equidispersion. Values above 1.50 flag meaningful overdispersion that deflates Poisson standard errors, read next to NB2 alpha and the likelihood-ratio test.

⚠️ Common Misread to Avoid: Ignoring a high dispersion ratio because the Poisson log-count slopes look reasonable. Even when Poisson slopes are nearly unbiased, a dispersion ratio above 1.50 makes Poisson standard errors too small and inflates false positives.

📐 Technical Definition: Each term inside the sum is the squared Pearson residual. Dividing by residual degrees of freedom \(n - p\) centers the ratio near 1.00 when the variance law is properly specified.

🎯 The Manager's Question: How closely do {predictor} and {outcome} move together along a straight line?

📊 Worked Business Example: Across Brightline Retail's 48 weekly campaigns, ad spend and revenue correlate at r = 0.68. Weeks with higher spend tend to bring in higher revenue.

⚖️ How to Judge the Number: It runs from -1 (opposite directions) through 0 (no straight-line pattern) to +1 (lockstep). Read it next to R-squared (r squared) for shared variation and the scatterplot for shape or outliers.

⚠️ Common Misread to Avoid: Reading r = 0.68 as proof that raising {predictor} causes {outcome} to rise. Correlation is symmetric and blind to confounders: holiday demand could lift both spend and revenue at once.

📐 Technical Definition: Because each deviation is squared in the denominator, one extreme case far from the mean can dominate r. Confidence intervals use Fisher's z transform, \(z = \tfrac{1}{2}\ln\frac{1+r}{1-r}\), with standard error \(1/\sqrt{n-3}\).

🎯 The Manager's Question: Which inputs do the predictions lean on most, and which barely matter?

📊 Worked Business Example: At Ridgeline, shuffling emails_per_month across the 800 held-back subscribers raises the network’s log loss by 0.095, averaged over 5 shuffles. Shuffling tenure_months raises it by 0.041, so the predictions lean on emails a little over twice as hard.

⚖️ How to Judge the Number: 0 means the model ignores that input, and a small negative value is noise. Compare bars with each other, in the units of the lead score: at Ridgeline, 0.095 for emails against 0.007 for lifecycle stage. A bar whose spread over the shuffles reaches 0 is noise. Below 150 held-back rows the page warns that the ranking can change from one seed to the next.

⚠️ Common Misread to Avoid: A top bar is not a cause, and it has no direction. Emails rank first at Ridgeline, yet the effect is a U: churn is lowest near 4 or 5 emails a month and higher at both ends. The partial dependence curve shows that shape; the bar cannot.

📐 Technical Definition: Computed on the holdout rows, with every 0/1 column of a category shuffled together so the category moves as one input. The bar is the mean over the shuffles and the whisker their range. Correlated inputs share credit unpredictably, because a shuffle builds customers who cannot exist, such as a 40-month subscriber flagged as new.

🎯 The Manager's Question: Was the forecast right more often than simply repeating last period would have been?

📊 Worked Business Example: On Kestrel's 56 held-back days the model is right 87.5% of the time. Repeating yesterday is right 82.1%, because the shelf rarely changes state from one day to the next.

⚖️ How to Judge the Number: The bar is the "same as last period" row, not 50%. In a sticky series that rule is right most days, so beating it by even 5 points is real progress. Read it alongside switches caught, which shows whether the extra accuracy comes from seeing change.

⚠️ Common Misread to Avoid: Reading 87.5% accuracy as a strong model. Repeating yesterday scored 82.1% while knowing nothing about promotions or trucks, so the model adds about 5 points. Always read accuracy against that row.

📐 Technical Definition: Here \(\hat{y}_t = 1\) when the forecast is at least the threshold \(c\) (0.5 unless changed). The persistence rule forecasts \(\hat{y}_t = y_{t-1}\) and is scored the same way. It has no log loss or Brier score in the table, because it only ever forecasts 0% or 100%.

🎯 The Manager's Question: If we pretend the launch happened earlier, does the fake date also show a lift?

📊 Worked Business Example: In Hearthstone Coffee a fake launch at week 7 gives minus $36, about 1% of the real $3,230, so the placebo passes. In Pinnacle Fitness the fake date gives $1,054, 44% of the real estimate.

⚖️ How to Judge the Number: A placebo near 0 is reassuring, and the tool fails the test when it reaches 50% of the real estimate. Read it next to the placebo's own p-value as well. Pinnacle's 44% stays under the 50% line yet has p < 0.001, a clear sign the groups were already diverging.

⚠️ Common Misread to Avoid: Treating a large placebo effect as an interesting side finding. Nothing happened at the fake date, so any "lift" there shows the groups were already moving apart before the real launch. That same drift is sitting inside the real estimate.

📐 Technical Definition: The tool refits the two-way fixed-effects model on pre-launch periods only, with the fake date at the midpoint of the pre-period unless you choose another. It needs at least 3 pre-launch periods. It assumes nothing happened before the real launch, so anticipation can also make a placebo fail.

🎯 The Manager's Question: How many visitors per option does the A/B test need before it can decide?

📊 Worked Business Example: For Nordvik's heroes at 3.0% and 4.2%, the plan needs 3,782 visitors per option at 80% power and alpha 0.05. The test phase therefore uses 7,564 of the 20,000 visitors.

⚖️ How to Judge the Number: Read it against the traffic you have: here 2 × 3,782 = 7,564 of 20,000 visitors. If the plan needs more than the race provides, the test never finishes and ships the control. It grows with 1 over the gap squared, so halving the gap quadruples it.

⚠️ Common Misread to Avoid: Reading the plan as a target to stop at once results look good. Stopping early because a gap has appeared is the peeking problem. Checked often enough, a test of two identical pages names a winner far more than 5% of the time.

📐 Technical Definition: This is the two-proportion formula of the A/B sample-size calculator, at 80% power and alpha 0.05. With more than one variant, alpha is split across the comparisons with the control (Bonferroni) and the hardest comparison sets n. Equal options leave no gap to detect, so the test never finishes and ships the control.

🎯 The Manager's Question: What share of candidate SKUs currently sit on the shelf rather than being cut?

📊 Worked Business Example: In Cooler Wars, 17 of 18 energy drink SKUs start on the cooler shelves while 1 sits in the tray. That gives an active share of 94.4%.

⚖️ How to Judge the Number: 100% means every candidate SKU is listed on the fixture. Read it alongside the Delist What-If walkout rate and Days of Supply before cutting a slow mover.

⚠️ Common Misread to Avoid: Cutting every bottom-decile SKU solely because its weekly unit volume is small. Check the walkout rate first, because loyal buyers of a specialty item often leave the store with their entire basket.

📐 Technical Definition: A SKU is listed when it holds at least one horizontal facing on an unlocked shelf. Unplaced SKUs remain in the assortment tray with zero facings and earn zero unit sales.

🎯 The Manager's Question: How many days will a full shelf block last at historical daily sales velocity?

📊 Worked Business Example: A SKU with 20 units of shelf capacity selling 28 units per week (4.0 units per day) has 5.0 Days of Supply. That clears a 4-day restock gap by 1.0 day.

⚖️ How to Judge the Number: Green requires at least 1.5x the longest restock gap (6.0 days for a 4-day gap) and 1 full case pack. Read it next to historical stockout days in the 52-week log.

⚠️ Common Misread to Avoid: Trusting a Green status light when past weekly sales were artificially capped by stockouts. If a SKU stocked out 3 days a week last year, true daily demand is higher and the shelf will empty faster.

📐 Technical Definition: Dividing weekly average units \(\bar{u}_i\) by 7 converts weekly velocity into a daily burn rate. Blocks with \(\text{DOS}_i\) below the longest delivery gap trigger a Red stockout warning.

🎯 The Manager's Question: How many packages fit front-to-back in a single facing column on the shelf?

📊 Worked Business Example: In a 26-inch deep cooler, a 16 oz can with 2.6-inch depth fits 10 cans deep per facing. A wider 24 oz can with 3.0-inch depth fits 8 cans deep.

⚖️ How to Judge the Number: 8 to 11 units deep is standard for beverage coolers, while large kibble bags fit only 2 units deep. Read it alongside the distributor case pack size and shelf capacity.

⚠️ Common Misread to Avoid: Dividing fixture depth by package depth without rounding down to a whole integer. A 26-inch shelf divided by a 3.0-inch can equals 8.67, which holds 8 cans deep rather than 9.

📐 Technical Definition: Physical packages cannot overhang the front lip or crush against the rear wall. The floor function drops any fractional remainder behind the last full unit in the lane.

🎯 The Manager's Question: Does a brand earn a larger or smaller share of category revenue than its share of shelf width?

📊 Worked Business Example: If Plainleaf generates 14.5% of category dollar sales while occupying 9.5% of linear shelf width, its Fair-Share Index is 1.53. That flags the brand as under-spaced relative to parity (1.00).

⚖️ How to Judge the Number: 1.00 marks exact parity between revenue share and width share. Values above 1.08 flag under-spaced brands and values below 0.92 flag over-spaced brands, read alongside Profit per Foot.

⚠️ Common Misread to Avoid: Forcing every brand to a 1.00 Fair-Share Index without checking unit margins or stockout censoring. A high-margin or stockout-capped brand may warrant space above historical sales parity.

📐 Technical Definition: Inverts the textbook Space-to-Sales Index so values above 1.00 highlight brands pulling more revenue share than their allocated footage. Always cross-check against gross margin share before reallocating facings.

🎯 The Manager's Question: How many annualized gross profit dollars does each dollar of average shelf inventory earn?

📊 Worked Business Example: A SKU earning $2,369 in annual gross margin while holding 20 units at $1.95 cost ($19.50 average inventory) achieves a GMROI of 121.5x. Across the whole cooler, baseline GMROI is 51.3x.

⚖️ How to Judge the Number: Compare SKU GMROI against the category benchmark (51.3x in Cooler Wars). Read it alongside Days of Supply to protect shelf availability.

⚠️ Common Misread to Avoid: Maximizing GMROI by starving high-velocity items of facings to shrink average inventory cost. Once shelf capacity drops below 4 days of supply, lost stockout sales destroy total dollar profit.

📐 Technical Definition: Annualizes weekly gross margin by multiplying by 52 and divides by average shelf inventory valued at wholesale cost \((K_i / 2) c_i\). Over-facing a slow mover inflates \(K_i\) and drags GMROI down.

🎯 The Manager's Question: How many weekly gross profit dollars does each linear foot of occupied shelf space return?

📊 Worked Business Example: A SKU selling 34 units per week at a $1.34 margin earns $45.56 in weekly profit. Across 0.433 linear feet, that equals $105.14 per linear foot.

⚖️ How to Judge the Number: Compare each SKU against the category mean ($40.48 per foot in Cooler Wars). Read it alongside space elasticity and shelf zone multipliers.

⚠️ Common Misread to Avoid: Treating average historical profit per foot as the marginal gain from adding another facing. Because space elasticity is around 0.15 to 0.25, each extra facing adds much less profit than the first facing.

📐 Technical Definition: Multiplies weekly unit sales by unit dollar margin \(m_i = p_i - c_i\) and divides by occupied linear feet. Eye-level shelves should prioritize items with the highest marginal profit per linear inch.

🎯 The Manager's Question: What weekly dollar revenue would the listed SKUs earn at last year's unit velocity?

📊 Worked Business Example: In Cooler Wars, the 17 listed SKUs average 230.2 units per week across prices from $2.29 to $3.79. Multiplying each SKU's historical units by price sums to $758.80 per week.

⚖️ How to Judge the Number: Compare your plan against last year's baseline ($758.80 in Cooler Wars). Read it alongside Profit per Foot and Delist What-If.

⚠️ Common Misread to Avoid: Expecting baseline projected sales to jump automatically when you add a facing on the canvas. Live baseline metrics hold historical velocity fixed so you use the diagnostic lab to separate real facing lift from stockout recovery.

📐 Technical Definition: Sums historical weekly revenue across all currently listed SKUs while assigning zero revenue to delisted items. For new launches without history, the supplier's launch estimate is substituted and flagged.

🎯 The Manager's Question: How many weekly sales dollars does each linear foot of occupied shelf space generate?

📊 Worked Business Example: A SKU generating $111.86 per week across two 2.6-inch facings (5.2 inches, or 0.433 feet) earns $258.14 per linear foot. Across 8.05 occupied feet, the cooler averages $94.26 per foot.

⚖️ How to Judge the Number: Compare each SKU against the category average ($94.26 per foot in Cooler Wars). Read it alongside Profit per Foot and Days of Supply.

⚠️ Common Misread to Avoid: Cutting every SKU down to 1 facing to inflate sales per linear foot. Squeezing a fast mover into 1 facing spikes revenue per inch on paper but causes chronic mid-week stockouts.

📐 Technical Definition: Normalizes weekly dollar turnover by linear footage so narrow cans and wide bags can be compared on a common physical footprint. Category-level figures report productivity per occupied foot and per available foot.

🎯 The Manager's Question: How many total units of a SKU can sit on the shelf when fully restocked?

📊 Worked Business Example: At 2 horizontal facings, 1 unit high, and 10 cans deep, Voltfang Original holds 20 cans when full. Adding a third facing raises capacity to 30 cans.

⚖️ How to Judge the Number: Capacity should at least clear 1 distributor case pack (12 or 24 cans) so clerks can stock a full case. Read it alongside Days of Supply and the longest restock gap.

⚠️ Common Misread to Avoid: Counting front facings alone without checking total units held in the lane. A 2-facing slim can block holds 22 cans in a 26-inch cooler, which still falls short of a 24-can flat.

📐 Technical Definition: Upon each restock delivery, inventory on the shelf is replenished up to \(K_i\) units. Any customer demand arriving after those \(K_i\) units sell out is censored or lost until the next delivery day.

🎯 The Manager's Question: By what percentage do unit sales rise when a SKU's facings increase by one percent?

📊 Worked Business Example: At a controlled space elasticity of 0.18, doubling a SKU from 2 to 4 facings (+100%) lifts unit demand by 13.3%. It does not double unit sales.

⚖️ How to Judge the Number: Published retail meta-analyses place true space elasticity near 0.15 to 0.25 (Eisend, 2014). Read the Controlled Fixed-Effects estimate next to the Naive Pooled OLS slope to spot endogeneity bias.

⚠️ Common Misread to Avoid: Running a naive pooled regression of log units on log facings across all SKUs and believing a slope near 0.65. Popular SKUs already sit at eye level with extra facings, so control for SKU, zone, promos, and stockouts.

📐 Technical Definition: Because \(0 < \beta < 1\), marginal returns to shelf space diminish rapidly. Estimating \(\beta\) accurately requires filtering out stockout weeks and promos while absorbing SKU and zone fixed effects.

🎯 The Manager's Question: What fraction of available horizontal shelf width is currently occupied by products?

📊 Worked Business Example: In Cooler Wars, 17 placed SKUs occupy 96.6 inches across 120.0 available shelf inches. That equals 80.5% linear space utilization (8.05 of 10.00 linear feet).

⚖️ How to Judge the Number: 90% to 98% utilization puts almost all unlocked footage to work without jamming the grid. Read it next to Days of Supply and Gross Profit per Foot.

⚠️ Common Misread to Avoid: Treating all occupied inches as equally valuable across the fixture. An inch on an eye-level shelf carries a 1.25x attention multiplier, whereas a bottom-shelf inch carries 0.75x.

📐 Technical Definition: Each placed SKU consumes horizontal width equal to its facing count multiplied by package width \(w_i\). Category utilization sums occupied inches across all unlocked shelves and divides by total available linear inches.

🎯 The Manager's Question: Of the people who clicked the ad, what share went on to buy?

📊 Worked Business Example: At 2%, Plan A's 10,000 clicks turn into 200 credited purchases. A weaker landing page at 1% halves that to 100 and doubles the cost per purchase from $50 to $100.

⚖️ How to Judge the Number: 0% means no click led to a credited purchase. The tool's baseline is 2%, so 1% is half. Read it next to the click rate, because the two multiply: 1% of clicks at a 1% click rate equals 2% of clicks at 0.5%.

⚠️ Common Misread to Avoid: Reading 2% as 2% of everyone who saw the ad. The denominator is clicks: 2% of 10,000 clicks is 200 purchases, which is only 0.01% of the 2,000,000 impressions.

📐 Technical Definition: Each click is modeled as leading to at most one credited purchase. The tool computes expected purchases = clicks × CVR ÷ 100.

🎯 The Manager's Question: When the model flags a customer, how likely is it that the flag is right?

📊 Worked Business Example: In the churn preset the model flags 116 of 1,000 subscribers, and 18 of them really are about to cancel. Precision is 18 / 116, about 15.5%, so 5 of every 6 retention credits go to customers who were staying anyway.

⚖️ How to Judge the Number: The anchor is the base rate: a model that flags at random has precision equal to it, here 2%. Above 50%, most alerts are real; at 15.5%, more than 4 in 5 are false alarms. Read it alongside sensitivity, the share of real cases caught, and decide whether each alert is worth the cost of acting on it.

⚠️ Common Misread to Avoid: Reading 90% sensitivity as "90% of alerts are right". Sensitivity starts from the real churners and asks how many get flagged; precision starts from the flags and asks how many are real. At a 2% base rate the same model has 90% sensitivity and 15.5% precision.

📐 Technical Definition: The second form is Bayes' rule, which is why one model gives different precision in groups with different base rates. The lab computes it exactly from its three sliders and shows the counts for 1,000 customers rounded to whole people. It matches scikit-learn's precision_score on the same confusion matrix.

🎯 The Manager's Question: When the model flags a customer, how likely is it that the flag is right?

📊 Worked Business Example: In the churn preset the model flags 116 of 1,000 subscribers, and 18 of them really are about to cancel. Precision is 18 / 116, about 15.5%, so 5 of every 6 retention credits go to customers who were staying anyway.

⚖️ How to Judge the Number: The anchor is the base rate: a model that flags at random has precision equal to it, here 2%. Above 50%, most alerts are real; at 15.5%, more than 4 in 5 are false alarms. Read it alongside sensitivity, the share of real cases caught, and decide whether each alert is worth the cost of acting on it.

⚠️ Common Misread to Avoid: Reading 90% sensitivity as "90% of alerts are right". Sensitivity starts from the real churners and asks how many get flagged; precision starts from the flags and asks how many are real. At a 2% base rate the same model has 90% sensitivity and 15.5% precision.

📐 Technical Definition: The second form is Bayes' rule, which is why one model gives different precision in groups with different base rates. The lab computes it exactly from its three sliders and shows the counts for 1,000 customers rounded to whole people. It matches scikit-learn's precision_score on the same confusion matrix.

🎯 The Manager's Question: Of the customers the model flags, what share really turn out to be yes?

📊 Worked Business Example: At the 0.50 cutoff in the influencer preset the model flags 361 posts and 277 of them converted, a precision of 76.7%. Raise the cutoff to 0.70 and precision climbs to 89.4%, but only 217 posts are flagged.

⚖️ How to Judge the Number: Flagging at random gives a precision equal to the base rate, 46.3% here, so that is the floor to beat. Higher means fewer wasted contacts per real yes. Read it together with sensitivity, because a stricter cutoff buys precision by missing more of the yes customers: 59.8% are caught at 0.50 and 41.9% at 0.70.

⚠️ Common Misread to Avoid: Reading precision as the share of yes customers the model caught. That is sensitivity, which divides by actual yeses, not by flagged customers. Here 76.7% of flagged posts converted, yet only 59.8% of converting posts were flagged.

📐 Technical Definition: It cannot be computed when nothing is flagged. It depends on the cutoff and on how common the outcome is, so the same model shows lower precision in a market with a lower base rate. It equals 1 minus the false discovery rate.

🎯 The Manager's Question: Which outcome should we act on for this customer?

📊 Worked Business Example: At the default 0.50 cutoff the influencer model labels 361 of 1,000 posts as converting. Lower the cutoff to 0.30 and 642 posts get the label, catching 384 of the 463 that converted instead of 277.

⚖️ How to Judge the Number: For logistic regression it is the focal outcome once the probability reaches the cutoff set on the tool's slider, 0.50 unless you moved it. For multinomial models and trees it is the most likely category. Read it next to precision and sensitivity at that cutoff, and use the probability when you need to rank or budget.

⚠️ Common Misread to Avoid: Reading the label as the whole story. It hides how close the call was: 0.51 and 0.99 get the same label. And 0.50 is a convention, not a business decision; the right cutoff depends on what a missed customer and a wasted contact each cost.

📐 Technical Definition: With several categories the label is the category with the highest predicted probability; for a tree it is the majority category in the leaf. The confusion matrix, accuracy and sensitivity all use this label, so they change when the cutoff moves.

🎯 The Manager's Question: How many events does the calibrated model expect for a customer with this profile?

📊 Worked Business Example: In Mosaic Analytics, a baseline workspace observed for 8.1 active months has an expected rate of 0.62 tickets per month, producing 5.02 expected tickets over that window. At 12 active months, the same rate predicts 7.44 tickets.

⚖️ How to Judge the Number: Compare the predicted count against the sample mean count and the 95% confidence interval. When an exposure offset is active, read the total count together with the per-unit incidence rate.

⚠️ Common Misread to Avoid: Expecting every individual customer with that profile to land on the exact decimal prediction. Observed counts are whole numbers drawn around the expected mean, with spread governed by Poisson or NB2 variance.

📐 Technical Definition: Under Poisson and NB2, \(\hat{\mu}_i = E_i \exp(X_i\hat{\beta})\). Under a Two-Part Hurdle model, \(\hat{\mu}_i = \hat{\pi}_i \cdot \hat{\mu}_{2i} / (1 - \exp(-\hat{\mu}_{2i}))\).

🎯 The Manager's Question: How likely is this outcome for a customer like this one?

📊 Worked Business Example: In the influencer preset a Micro-tier US post with average views and 1,500 swipe-ups gets 0.85. Of 100 posts like it, about 85 should drive a sale if the model is well calibrated.

⚖️ How to Judge the Number: The no-model figure is the base rate given to everyone, 0.463 here. Read a score across many customers: of all those scored about 0.70, roughly 70% should turn out yes if the model is well calibrated. Read it next to its 95% range, and use it to rank customers or to multiply by a dollar value.

⚠️ Common Misread to Avoid: Reading 0.70 as "this customer will". Even when the model is well calibrated, three in ten customers scored 0.70 will not. It also does not say how sure the model is about its own estimate; the 95% range beside it does that.

📐 Technical Definition: For logistic regression \(\hat{\eta} = b_0 + b_1 x_1 + \dots\). A multinomial model uses a softmax over each category's score with the reference fixed at 0, so a row's probabilities add to 1. A tree uses its leaf rate instead.

🎯 The Manager's Question: What {outcome} should we expect for a customer with these characteristics?

📊 Worked Business Example: In the EDM event-spend scenario, a General Admission patron who preordered 30 days ahead has a predicted spend of about $53. That is the average for patrons like this one; one patron could land anywhere from about $31 to $75.

⚖️ How to Judge the Number: It is the model's best single guess, in {outcome}'s own units: the average for customers who look like this one. When the outcome was logged and converted back without a correction, it is the typical (middle) customer instead. Read it together with the 95% range on the same row, which shows how far one customer can land from it.

⚠️ Common Misread to Avoid: It is not what this customer will do. Individual customers scatter around it, often widely: the $53 patron above could spend $31 or $75. For a customer unlike anyone in the training data it is an extrapolation, and much less trustworthy than it looks.

📐 Technical Definition: That is the regression form; the Neural Network Lab gets its predicted value from the trained network instead. With a log-transformed outcome the model predicts on the log scale and converts back. A plain conversion, as in multiple regression's log option, estimates the typical (median) customer; the transformation lab's smearing correction estimates the average.

🎯 The Manager's Question: What value of {outcome} does the model expect for a case with exactly these predictor values?

📊 Worked Business Example: For a $50k campaign the line predicts 105.76 + 0.665 × 50, about $139.0k in revenue. The campaigns near $50k actually landed above that, because the true curve bulges in the middle.

⚖️ How to Judge the Number: It is in the units of {outcome}, and the gap between it and the actual value is that case's residual. Read it next to the residual standard error: if the misses are bell-shaped, about two thirds of cases land within 1 typical miss of their fitted value. Diagnostic plots put it along the bottom because it sums up all the predictors at once.

⚠️ Common Misread to Avoid: Treating it as a forecast for a new case. The model was fitted on this very case, so fitted values look more accurate than a forecast for new data would be. Holdout or cross-validation errors show the honest forecast accuracy.

📐 Technical Definition: With one predictor this is \(b_0 + b_1 x_i\), where \(b_0\) is the intercept and \(b_1\) the slope. Dummy columns add their gap from the baseline for the case's category. The fitted values are the points on the fitted line or surface.

🎯 The Manager's Question: Where should this one customer's actual {outcome} land?

📊 Worked Business Example: For a General Admission patron who preordered 30 days ahead, the EDM event-spend model predicts $53, with a 95% prediction interval of about $31 to $75. The confidence interval for the average such patron is only about $50 to $55.

⚖️ How to Judge the Number: If the model is right, about 19 customers in 20 land inside their own range. It is wide because it carries two uncertainties at once: customers scatter around the average, and the average itself was estimated from a sample. It grows for customers unlike the training data. Read it against the held-back rows, where "Rows inside their 95% range" shows whether the ranges held up. Plan for the ends, not the middle. If the range is too wide to act on, only better predictors narrow it; more rows of the same data barely do, because s does not shrink with n.

⚠️ Common Misread to Avoid: It is not the range for the average customer like this one; that is the confidence interval, which is much narrower. For the 30-day General Admission patron, the average sits between about $50 and $55; one patron, between $31 and $75.

📐 Technical Definition: The \(x_0^\top (X^\top X)^{-1} x_0\) term grows as the row moves away from the training data, which is why the range widens there. With a transformed outcome the range is built on the transformed scale and both ends are converted back. So it is lopsided, usually longer on the high side. When one end has no value in real units, the range is left blank rather than clipped; a square-root model whose low end is below zero is an example.

🎯 The Manager's Question: If we change one feature of our product, how many points of preference do we win or lose?

📊 Worked Business Example: Adding storage moves the product's simulated preference share from 24.0% to 29.5%, a change of +5.5 points. Rivals and the no-buy option are held fixed.

⚖️ How to Judge the Number: 0 means the change wins no one over. Read it against the baseline share and next to the profit change in the same row: a gain in share can still lose money once unit cost rises.

⚠️ Common Misread to Avoid: Reading +5.5 points as a forecast of real market share. It is a shift in simulated preference among the products in the scenario only; distribution, awareness and promotions are left out.

📐 Technical Definition: Each share averages every respondent's logit choice probabilities in that scenario. Rivals and the outside option stay fixed, so the number answers one what-if question at a time.

🎯 The Manager's Question: Were test and control markets already drifting apart before the launch?

📊 Worked Business Example: In the Pinnacle Fitness preset the joint lead test gives p < 0.001 and the largest lead is 93% of the estimate, so the tool flags a violation. Hearthstone Coffee passes, with p = 0.13 and its largest lead at 16% of the effect.

⚖️ How to Judge the Number: The tool flags a violation when p is below 0.10 or any lead reaches 50% of the DiD estimate. A pass means this data shows no drift, which is necessary but not sufficient. Read it next to the event-study chart and the number of markets: with few markets, real drift can hide inside wide intervals.

⚠️ Common Misread to Avoid: Reading a pass as "parallel trends confirmed". A p-value above 0.10 only says this data cannot detect drift, and with few markets the test has little power. Hearthstone passes at p = 0.13, not far from failing.

📐 Technical Definition: The null hypothesis is that every lead \(\gamma_r\) with \(r \le -2\) is 0, with \(r = -1\) as the reference period. The p-value matches statsmodels' Wald test to three decimals on every preset.

🎯 The Manager's Question: How large an increase can the platform announce before departures start feeding on themselves?

📊 Worked Business Example: At the default 18%, about 8% of teams cancel at once and the run ends at a Revenue Index of 105.9. At 21% each team needs 5 of its 8 neighbors active instead of 4, and in this model the whole base unravels.

⚖️ How to Judge the Number: h = 0 is no change. At the announcement 0.45 × h of active teams cancel (8% at 18%), and each remaining team then needs τ + h/2 of its neighbors active. With 8 neighbors that need jumps a whole neighbor when τ + h/2 passes 50%, which is why 20% ends near 106 and 21% ends at 0. Read it next to the Revenue Index. The cliff's position belongs to the model's settings: at τ = 35% it falls between 30% and 31%, and at r = 2 between 25% and 30%. Replications will not reveal this, because every seed collapses at 21% and none at 20%; vary τ and r instead.

⚠️ Common Misread to Avoid: Forecasting the result from the immediate cancellations alone. Each team that leaves makes the product less useful to its neighbors. At 18% that adds 1.3 points of extra losses; at 21% it unravels the whole base.

📐 Technical Definition: At the announcement each active team cancels with chance \(\min(0.9, 0.45h)\), and the price multiplier becomes \(1 + h\). These response rules are assumptions of the model; at the default they match the pilot survey's 8%.

🎯 The Manager's Question: How strongly does this survey item reflect the idea the factor stands for?

📊 Worked Business Example: In the default one-factor solution, "Support green policies" loads 0.65 and "Watch sports on weekend" loads 0.03. The first is a good marker of environmental concern; the second measures something else entirely.

⚖️ How to Judge the Number: Loadings run from -1 to 1, and 0 means no link to the factor. The tool calls a primary loading below 0.40 weak, and such an item is a candidate to drop or rewrite; the chapter calls 0.70 and up strong. Read it next to the item's communality and its second-highest loading, since an item can load 0.50 and still cross-load.

⚠️ Common Misread to Avoid: Reading a loading of 0.60 as "the item explains 60% of the factor". On a single factor a loading is the item's correlation with the factor, so its square is the share of the item the factor explains. Here that is 0.60 squared, 0.36, about a third.

📐 Technical Definition: The tool extracts factors by principal axis factoring, with principal components as the alternative. It then rotates with Promax, which lets factors correlate. The loadings shown are the pattern matrix: each is a regression weight with the other factors held fixed, not a plain correlation.

🎯 The Manager's Question: Before any model flags anyone, what share of our customers actually has the event?

📊 Worked Business Example: In the churn preset, 2% of subscribers cancel in a month: 20 of every 1,000. That 2% is the prior, and it caps how trustworthy any churn alert can be.

⚖️ How to Judge the Number: There is no neutral value: the rarer the event, the more false alarms swamp the real cases. At a 10% base rate, a model with 90% sensitivity and 90% specificity is right on half its alerts; at 2%, on about 1 in 6. Read it next to the false-alarm rate, then choose between scoring everyone and scoring a higher-risk group.

⚠️ Common Misread to Avoid: Mixing up the base rate with the model's accuracy. A 90%-accurate churn model says nothing about how common churn is, and at a 2% base rate it is right on only 18 of its 116 alerts. Raising the base rate by pre-screening helps more than polishing the model.

📐 Technical Definition: It is the marginal probability of the event, before any signal is seen. In Bayes' rule it weights the true-positive side: \(P(A \mid B) \propto P(B \mid A)\,P(A)\). The lab sets it with the prior slider and applies it to 1,000 simulated customers.

🎯 The Manager's Question: If I pick one case from each group at random, how often is the {groupB} one higher?

📊 Worked Business Example: In the whale-order case, a random Flow B order beats a random Flow A order 64% of the time. The means point the other way, $113 for Flow A against $101 for Flow B, because of one $4,120 bulk order.

⚖️ How to Judge the Number: 50% means no tendency either way and 100% means every {groupB} case beats every {groupA} case. On the bands below, 0.64 sits right at the start of the moderate range. Read it next to the Mann-Whitney p-value and the medians. Then match it to the decision: for the typical order use PS and the medians; for total revenue use the means.

⚠️ Common Misread to Avoid: Reading 0.64 as "Flow B orders are 64% bigger". PS says nothing about size: it is the share of random pairings, one case from each group, that {groupB} wins. The size gap is a separate number; in the whale-order case the medians differ by 42%, $89.39 against $62.80.

📐 Technical Definition: It equals \(U_B / (n_A n_B)\), the Mann-Whitney U for {groupB} divided by the number of pairs, so it needs no assumption about the shape of either distribution. Ties count half to each side, which is why data full of ties lands near 0.5.

🎯 The Manager's Question: What share of this segment takes the action, the rate a campaign forecast starts from?

📊 Worked Business Example: 120 of the 1,000 people in Segment A convert, so the probability is 120 / 1,000 = 0.12, or 12 in every 100. In Segment B it is 40 / 1,000 = 0.04.

⚖️ How to Judge the Number: 0 means it never happens, 1 means it always does, and 0.5 is an even split. The scale is absolute, so 0.12 against 0.04 is a gap of 8 percentage points. Read it next to the other group's probability: that gap in points is what a campaign budget can be costed against.

⚠️ Common Misread to Avoid: Reading a probability of 0.12 as odds of 0.12. Probability counts the people with the outcome out of everyone, 120 of 1,000. Odds count them against the people without it, 120 to 880, which is 0.136.

📐 Technical Definition: In a 2x2 table, group A's probability is \(a/(a+b)\) and group B's is \(c/(c+d)\), each divided by its own row total. A sample probability estimates the true rate, with a standard error of about \(\sqrt{P(1-P)/n}\). Odds-O-Matic reads it from the table, or from the sliders when you switch to probability inputs.

🎯 The Manager's Question: How precisely does the model know the chance for customers like this one?

📊 Worked Business Example: A Micro-tier US post with 1,500 swipe-ups gets 0.85 with a range of 0.78 to 0.90. A Mega-tier LATAM post with 100,000 views and 300 swipe-ups gets 0.21, but its range runs from 0.06 to 0.52 because few posts look like it.

⚖️ How to Judge the Number: A range that spans 0.50 cannot even say which side of a coin flip the customer is on. Narrow means plenty of similar customers informed the estimate; wide means few did. Two customers can share a 0.60 prediction with very different ranges, so read it next to the prediction and trust the narrow one more.

⚠️ Common Misread to Avoid: It is not a range for what this one customer will do. A yes-or-no outcome has no such range: it happens or it does not, so do not check whether actual outcomes "fall inside" it. It also leaves out the uncertainty from a predictor the model lacks.

📐 Technical Definition: A Wald interval built on the log-odds scale with both ends converted back, so the range stays between 0 and 1 and is lopsided near the edges. For a multinomial model the delta method gives each category's standard error on its own log-odds.

🎯 The Manager's Question: How different are the predicted chances for two customer profiles that differ in only one thing?

📊 Worked Business Example: In the influencer data, a Micro-tier US post with story views at their average and 500 swipe-ups has a predicted 41.8% chance of a sale; with 1,000 swipe-ups it has 66.7%. B minus A is 24.9 percentage points.

⚖️ How to Judge the Number: 0 means the change makes no difference to the predicted chance. B minus A is positive when B is more likely and negative when it is less. The same change moves a profile near 50% far more than one near 5% or 95%, so read it with the starting probability. Weigh the size against the decision, such as the cost of earning 500 more swipe-ups.

⚠️ Common Misread to Avoid: Reading a move from 20% to 30% as "10% higher". It is 10 percentage points, which is a 50% relative increase. The tool reports points, so quote it as points.

📐 Technical Definition: Both probabilities use the same fitted coefficients and the same predictor coding, and the profiles differ only in the predictor chosen. It is a fitted association, not the effect of an intervention. The separate intervals on each probability do not give an interval or a test for the difference.

🎯 The Manager's Question: Which factor does each item belong to, once the overlap between factors is taken out?

📊 Worked Business Example: In PureGlow, "Truthful ingredient labeling" has a pattern loading of 0.80 on Authenticity and near 0 on the others, so it belongs to Authenticity alone. Its structure correlation with Efficacy is still 0.29, because those two factors correlate 0.28.

⚖️ How to Judge the Number: Read each row for one big loading and the rest near 0; the big one names the item's factor, and 0.40 is the tool's line for a clear loading. Read it alongside the factor correlations. When factors correlate 0.30 or more, pattern and structure loadings differ, and the pattern matrix is the one for assigning items.

⚠️ Common Misread to Avoid: Reading a pattern loading as the item's correlation with the factor. After Promax it is a regression weight with the other factors held fixed, the item's unique link. The plain correlations are in the structure matrix, and they run larger whenever factors correlate.

📐 Technical Definition: After an oblique rotation the factors correlate, \(\Phi = E[\mathbf{f}\mathbf{f}^{\top}] \ne I\), so each item needs two matrices: the pattern \(\Lambda\) and the structure \(S = \Lambda \Phi\). The tool runs Promax (power 4) after principal axis factoring, then sorts the factors and flips signs so the largest loadings are positive.

🎯 The Manager's Question: How much does the publisher earn from ads for every 1,000 page visits?

📊 Worked Business Example: An advertiser pays $5 CPM and the selling platform keeps a 20% fee, so the publisher nets $4 per 1,000 billable impressions. If all three spaces on the page sell at that price, page RPM is 3 × $4 = $12.

⚖️ How to Judge the Number: It is $0 when no space sells, and it rises with the price, the share of spaces filled and the number of spaces per page. Compare it only against figures on the same revenue basis, before or after fees. Read it next to the reader experience: a fourth ad space can lift it while driving readers away.

⚠️ Common Misread to Avoid: Reading page RPM as the CPM an advertiser pays. A $5 CPM to the buyer is $4 to the publisher after a 20% fee, and page RPM then adds up every ad space on the page.

📐 Technical Definition: The 20% fee is the simulator's illustrative assumption, not a market average. Its campaign model leaves out the publisher's other advertisers, unsold spaces and costs.

🎯 The Manager's Question: Do the model's misses follow a bell curve, or do big misses happen more often than one allows?

📊 Worked Business Example: In the 60 ad campaigns, the lowest points fall well below the line: the three smallest budgets missed by $15k to $22k on the low side. That sag at the bottom is the long left tail behind a skewness of -0.86.

⚖️ How to Judge the Number: Points should hug the diagonal from end to end. A few points peeling away at one end are single extreme cases; an S shape means the tails are heavier or lighter than a bell curve. Read it together with the Shapiro-Wilk p-value (0.017 here) and the sample size: with 30 or more rows, mild bends matter little.

⚠️ Common Misread to Avoid: Checking the raw outcome instead of the residuals. Sales revenue can be skewed while the misses are bell-shaped, because the predictor accounts for the skew. The plot is about the misses only.

📐 Technical Definition: Each ordered standardized residual \(r_{(i)}\) is plotted against \(q_i\), where \(r_{(i)}\) is the ith smallest standardized residual. The fractions \((i-0.375)/(n+0.25)\) are Blom's plotting positions.

🎯 The Manager's Question: Does {outcome} bend as {predictor} grows, rather than rising or falling at a steady rate?

📊 Worked Business Example: Adding centred spend squared to the ad-campaign model gives a negative coefficient with t = -9.35 and p below 0.001. Negative means an arch: each extra $1k of spend buys less revenue than the last.

⚖️ How to Judge the Number: A Holm-adjusted p-value below {alpha} flags curvature in {predictor}. The sign gives the shape: negative is an arch, as with diminishing returns, and positive is a U. Read it alongside the plot of residuals against {predictor}, which shows how strong the bend is.

⚠️ Common Misread to Avoid: Treating curvature as a reason to drop {predictor}. It is a reason to model the bend, with a squared term or a log. Dropping a curved predictor throws away a real relationship.

📐 Technical Definition: Centring keeps the squared term from copying the straight one. The p-values are Holm-adjusted across the predictors tested, which holds the chance of any false flag at {alpha}.

🎯 The Manager's Question: Is model C's answer settled, or would a finer calculation change it?

📊 Worked Business Example: For Thistledown the tool fits model C with 12 nodes, refits with 18, and no estimate moves by more than 0.01%. That answer is settled.

⚖️ How to Judge the Number: Changes under 1% (or under 0.01 for values near 0) mean the answer is settled. A bigger change raises a warning: increase the nodes in the settings and run again. Read it alongside the warnings list, because an unsettled fit can move the habit effect and σ together.

⚠️ Common Misread to Avoid: Ignoring the warning because the numbers look plausible. An unsettled integral can shift every model C estimate at once, habit and σ included. Raise the nodes until the check passes before quoting anything from model C.

📐 Technical Definition: Adaptive Gauss-Hermite quadrature centres and scales the nodes on each customer's posterior mode. The check refits at \(1.5K\) nodes, starting from the converged values. It flags a coefficient, or σ, that moves by more than the larger of 1% of its value and 0.01.

🎯 The Manager's Question: How often do two qualitative codes appear together within the same participant utterance?

📊 Worked Business Example: "Packaging Frustration" and "Brand Switching" are tagged together on 9 transcript segments. Their co-occurrence count is 9.

⚖️ How to Judge the Number: A count of 0 means the two themes never appear in the same turn, read alongside each code’s total frequency.

⚠️ Common Misread to Avoid: Treating two codes that frequently co-occur as redundant labels. Co-occurrence often reveals a substantive link where a specific pain point triggers a behavioral consequence.

📐 Technical Definition: Off-diagonal cells of the co-occurrence matrix highlight conceptual bridges. They link functional product attributes to emotional or behavioral outcomes.

🎯 The Manager's Question: What share of transcript utterances or speaker turns has been tagged with this qualitative code?

📊 Worked Business Example: In a 60-turn focus group transcript, 15 turns are tagged with "Price Sensitivity". That gives a code coverage rate of 25%.

⚖️ How to Judge the Number: Compare coverage from 0% to 100% across codes alongside speaker-level breakdowns and verbatim quotes.

⚠️ Common Misread to Avoid: Equating high utterance count with consensus across participants. One vocal participant repeating the same point ten times inflates turn counts unless you check speaker distribution.

📐 Technical Definition: Systematic content analysis tracks both utterance-level coverage and unique-speaker coverage. That prevents one vocal speaker from distorting qualitative summaries.

🎯 The Manager's Question: Which part of a search ad, the ad text, the keyword or the landing page, should we fix first?

📊 Worked Business Example: A keyword shows 4 out of 10, with expected click-through rate rated "below average" and the landing page "average". The diagnostic points at the ad copy first; it does not set the price in the next auction.

⚖️ How to Judge the Number: It runs from 1 to 10 and compares this keyword's ads with other advertisers' ads on the same keyword. A 3 says they trail most competitors; a 9 says they lead. Read it next to its three components (expected click-through rate, ad relevance, landing page experience), and fix the weakest one.

⚠️ Common Misread to Avoid: Reading the displayed score as the number Google plugs into the auction, so that 5 to 8 cuts cost per click by a set amount. Google states the displayed score is not an auction input. Better components still tend to lower cost per click, because the same signals feed auction-time quality; what is wrong is a fixed conversion from the displayed score.

📐 Technical Definition: A keyword-level summary of three historical components, each rated above average, average or below average. Google Ads reports it, and no general payment formula follows from it. The chapter's Auction Explorer uses bid × Quality Score as a teaching model, not as Google's rule.

🎯 The Manager's Question: How much of the ups and downs in {outcome} does this model account for?

📊 Worked Business Example: Fitting sales revenue on ad spend across the residual diagnostics tool's 60 campaigns gives R² = 0.87. The straight line accounts for 87% of the campaign-to-campaign swings in revenue.

⚖️ How to Judge the Number: 0 means the model accounts for none of the variation in {outcome} and 1 means all of it. What counts as good depends on the data: 0.2 can be strong for individual customers, while aggregate sales often reach 0.8. Read it next to the residual plots, which show whether the line has the right shape.

⚠️ Common Misread to Avoid: Reading a high R² as a sign that the straight line is the right model. The 60 campaigns give 0.87, yet revenue flattens at high spend, so the line overstates what the biggest budgets return. R² says how much is explained; the residual plots say whether the shape is right.

📐 Technical Definition: On the ad-spend data \(SS_{res}\) is about 3,060 against \(SS_{tot}\) of about 23,000, so \(R^2 = 1 - 0.13 = 0.87\). It is computed on the rows the model was fitted to, so it flatters the model on new data. It never falls when a predictor is added, however useless, which is why adjusted \(R^2\) exists.

🎯 The Manager's Question: Is there a bend in the relationship that a straight-line model is missing?

📊 Worked Business Example: For the 60 ad campaigns RESET gives F = 87.4 with p below 0.001. If the relationship were truly straight, a bend this clear would almost never turn up.

⚖️ How to Judge the Number: A p-value below {alpha} flags a bend. Read it next to the residuals-against-fitted plot to see its shape, and the squared-term test for each predictor to see where it is. A bend means the slope averages different rates, so model the curve before you budget off it.

⚠️ Common Misread to Avoid: Reading a pass as proof the relationship is straight. Small samples can miss a real bend. A high R² says nothing either way: the ad campaigns have an R² of 0.87 and still fail RESET.

📐 Technical Definition: The tool tests \(\gamma = 0\) with a t-test on \(n-k-1\) degrees of freedom, where n is the number of cases and k the number of coefficients in the original model. Its F is that t squared, matching statsmodels linear_reset with power 2. Some packages also add a cubed term.

🎯 The Manager's Question: How much do customers' usual buying rates differ beyond what the data records about them?

📊 Worked Business Example: Thistledown's σ is 1.32 (interval 1.21 to 1.43) on the log-odds scale. Moving a typical shopper's baseline up by one σ multiplies their odds of buying by about 3.7.

⚖️ How to Judge the Number: 0 means customers differ by nothing beyond the model's terms, and model C collapses to model B. Saltmarsh Books shows 0.06; Thistledown shows 1.32, where one σ multiplies the odds by 3.7. A σ well above 0 means quote model C's habit effect and trial value, not model A's. Read it next to the latent ICC and the test that customers differ.

⚠️ Common Misread to Avoid: Reading Thistledown's σ of 1.32 as customers differing by 1.32 points, or by 132%, in their chance of buying. It is on the log-odds scale: one σ multiplies the odds of buying by about 3.7. Together with what the data records, it spreads the middle 80% of Thistledown shoppers from 1.8% to 43.9%.

📐 Technical Definition: Estimated with the coefficients by maximum likelihood, using adaptive Gauss-Hermite quadrature at 12 nodes by default. The interval is built on the log scale, so it is not symmetric around the estimate. Because σ cannot go below 0, the tool halves the p-value of the test that customers differ.

🎯 The Manager's Question: Which part of repeat buying is habit, and which part is simply who the customer is?

📊 Worked Business Example: Model C gives each Thistledown shopper their own baseline and learns that baselines spread with a standard deviation of 1.32 on the log-odds scale. With that in place, last week's purchase adds 4.2 points instead of 29.9.

⚖️ How to Judge the Number: Read "random effects" as "each customer gets their own baseline, and the model learns how spread out those baselines are". A between-customer SD of 0 means model C collapses to model B. Read it against model A: the gap between the two is what customers' differences were doing.

⚠️ Common Misread to Avoid: Reading "random" as haphazard or randomly assigned. Nothing was assigned at random, and the estimates are not noisier for the name. It means only that customer baselines are treated as draws from a bell curve, so the model estimates its width instead of 500 separate numbers.

📐 Technical Definition: The \(u_i\) are not parameters: the likelihood integrates them out, \(L_i = \int \prod_t P(y_{it} \mid u)\, \phi(u; 0, \sigma^2)\, du\), so only \(\beta\) and \(\sigma\) are estimated. Each customer's baseline is then read from their history as a posterior pulled toward the average. The model assumes baselines are unrelated to the drivers once the first-period and average terms are in. Fixed effects is not the safer option: a dummy per customer is biased when last period's outcome is in the model and histories are short.

🎯 The Manager's Question: If the {treatment} had gone to a random set of markets, how often would the lift look this big?

📊 Worked Business Example: The app-launch data has 495 ways to pick 4 test markets from 12. Only the real assignment gives a lift as large as $30K, so p = 1/495, about 0.002.

⚖️ How to Judge the Number: Below {alpha}, few random assignments match the real lift. The result is then hard to put down to which markets happened to be picked. With 3 of 6 markets treated there are only 20 assignments and the mirror-image split always ties, so p cannot go below 0.10 and the test can never clear 0.05. Read it next to the cluster-robust p-value; when markets are few, trust this one more.

⚠️ Common Misread to Avoid: Reading it as the probability that the {treatment} worked. It is how often a random choice of test markets would give a lift at least this large if the {treatment} did nothing. A p of 0.002 says the real assignment is unusual, not that the lift is big enough to pay.

📐 Technical Definition: The tool reshuffles which markets count as treated, keeping the number treated fixed, and recomputes the 2×2 DiD from each market's pre-to-post change. It needs no normal approximation, so it holds up with few markets, but it assumes any set of markets could have been the treated one. With random draws, a p of 0 means fewer than 1 in 2,000, not zero.

🎯 The Manager's Question: How consistently does one side come out ahead, on a scale from -1 to 1?

📊 Worked Business Example: After Mei Lin's app redesign, 39 of 64 users scored higher, 14 lower and 11 the same. Weighting each change by the rank of its size gives r = 0.58: the rises clearly outweigh the falls.

⚖️ How to Judge the Number: 0 means the rises and falls, or the two groups, balance exactly; 1 would mean every comparison goes the same way. Read it next to the counts of cases that went up and down, and the p-value.

⚠️ Common Misread to Avoid: Reading r = 0.58 as a correlation between the before and after scores. It measures how lopsided the changes are: the rank sum of the rises minus that of the falls, over their total. Users who did not change are left out of it.

📐 Technical Definition: Differences are ranked by size with zeros dropped, which is Wilcoxon’s method. For two independent groups the tool reports \(r = 2\,PS - 1\), positive when the second group ranks higher: the probability of superiority stretched onto a -1 to 1 scale.

🎯 The Manager's Question: Does the model rank the customers who said yes above the ones who said no?

📊 Worked Business Example: The influencer model has an AUC of 0.79: pick one post that converted and one that did not, and 79 times in 100 the converting post got the higher score. A coin toss would manage 50.

⚖️ How to Judge the Number: 0.5 is a random ranking and 1 is perfect. Hosmer and Lemeshow call 0.7 to 0.8 acceptable, 0.8 to 0.9 excellent and above 0.9 outstanding, so 0.79 is solidly acceptable. Read the training figure against the holdout figure in Validation & scoring: a large drop means the model memorised.

⚠️ Common Misread to Avoid: Reading an AUC of 0.79 as 79% of customers classified correctly. It is about ranking pairs, and says nothing about whether the probabilities are right: scores that are all 20 points too high rank customers exactly as well, so the AUC does not move. On a rare outcome a high AUC can sit beside poor precision.

📐 Technical Definition: The tool counts every yes-no pair and scores a tie as half, which equals the area under the ROC curve and the Mann-Whitney U divided by the number of pairs. Its standard error, used for the holdout noise check, follows Hanley and McNeil (1982).

🎯 The Manager's Question: Does a category with only a handful of rows let those few rows steer part of the model?

📊 Worked Business Example: In the regional store preset, the West has only 6 of 120 stores, so each West store has leverage of at least 1/6, or 0.17. That is double the 0.083 cut-off, and the West coefficient rests on those 6 stores alone.

⚖️ How to Judge the Number: A category with m rows gives each of them leverage of at least 1/m, against a usual cut-off of 2k/n. A category with a single row has leverage 1: the model fits it exactly, so its residual is always 0. Read it alongside the residual spread for that category, which the tool skips below 5 rows.

⚠️ Common Misread to Avoid: Treating the rare category as noise you can ignore. Its coefficient rests on those few rows alone, so one unusual store can swing it. Report it as fragile, or merge the category with a similar one.

📐 Technical Definition: The bound holds because the 0/1 column for that category can be built from the design matrix, whichever level is the baseline. With \(m = 1\) it gives \(h_{ii} = 1\) and a residual of exactly 0, so the case cannot show up as a miss.

🎯 The Manager's Question: What share of {caseNounPlural} finds at least one thing they want in this set?

📊 Worked Business Example: In Frost & Fold's survey the three most popular flavors reach 59.7% of the 320 shoppers. The best three reach 85.3%, because swapping two chocolates for fruit flavors adds shoppers the chocolates missed.

⚖️ How to Judge the Number: There is no fixed good level: 60% can be excellent in a crowded category and poor in a narrow one. Judge it against the most-popular set of the same size; here 85.3% against 59.7% is a 25.6-point gain, far outside the 3.9-point near-tie margin. That gap is the case for launching the best set instead of the best sellers. Read it next to frequency.

⚠️ Common Misread to Avoid: Reading reach as sales. It counts people whose answers said at least one item appeals, which is stated intent, not purchase, frequency or price paid. Each person counts once however many of the items they like.

📐 Technical Definition: Respondents who accept nothing stay in the denominator. With weights, each respondent counts by their survey weight. The search scores every possible set while there are no more than 200,000 of them. Past that it builds the set greedily, then tries one pass of single swaps.

🎯 The Manager's Question: If we swap this item for its best alternative, do we gain or lose people?

📊 Worked Business Example: Swapping Chocolate Fudge Brownie for Chocolate Chip in Frost & Fold's best three changes reach by -2.2 points, from 85.3% to 83.1%. The best swap for either other flavor costs 0.6.

⚖️ How to Judge the Number: 0 means the swap leaves reach unchanged under the same acceptance rule, and a negative value means the current item is better. A change inside the near-tie margin, about 3.9 points here, is not a clear difference. Read it next to cost and supply, which may favour the swap anyway.

⚠️ Common Misread to Avoid: Reading -2.2 points as a 2.2% drop in reach. It is a subtraction of two percentages, 85.3% minus 83.1%. As a relative change it is a fall of about 2.6%.

📐 Technical Definition: The best replacement is searched over every unselected item in the pool, and a locked must-include item gets none. Both reaches use the same respondents, weights and acceptance rule.

🎯 The Manager's Question: Is the best set clearly better than simply launching the most popular items?

📊 Worked Business Example: Frost & Fold's best three reach 85.3% of shoppers against 59.7% for the three most popular, a gain of 25.6 points. The popular three are all chocolate and share one audience.

⚖️ How to Judge the Number: 0 means the most popular items are already the best set. That is a finding, not a failure: they do not compete for the same people. Read the gain against the near-tie margin, about 3.9 points here; a gain inside it is not a clear improvement.

⚠️ Common Misread to Avoid: Reading a large gain as proof that the popular items are bad. Each one is well liked; they overlap, so the second and third add few new people. A gain of 25.6 points says the overlap is the problem, not the flavors.

📐 Technical Definition: Without must-include or exclude constraints the gain is never negative, because the popularity set is one of the candidates the search scores. A rule under which nobody accepts more than one item makes reach additive, and the gain is then exactly 0.

🎯 The Manager's Question: Of every dollar shoppers paid this season, how much is left after product cost?

📊 Worked Business Example: NordicThreads starts at 39.9% with 40%-off codes running. By the time the run settles at Year 18 nearly everyone buys on deal and the margin is 8.3%.

⚖️ How to Judge the Number: 45% is the ceiling, when everyone pays the $50 list price against a $27.50 unit cost. A deal sale at 40% off keeps only 8.3%, and past 45% off each deal sale loses money. Read it next to Margin Leakage, which counts the points lost, and the share of shoppers buying on deal.

⚠️ Common Misread to Avoid: Judging the promotion by how many units it moves. In the default run the share of shoppers buying rises slightly while the margin falls from 39.9% to 8.3%. The margin falls because almost every sale is now made at $30.

📐 Technical Definition: Full-price buyers pay \(L = 50\) dollars, deal buyers pay \(L(1-d)\), and every unit costs \(0.55L\), or $27.50. Shoppers waiting for a sale buy nothing. The margin covers this season's purchases only.

🎯 The Manager's Question: How much more could the same total budget deliver if it were split differently across channels?

📊 Worked Business Example: Brightwell's default model moves about $2,400 a week into TV from search, social and display, adding about $1,860 a week on the same $25,600 budget. The tool shows this as +2.6%, measured against current weekly media contribution, not total sales.

⚖️ How to Judge the Number: 0% means the model finds no better split within the limits. Each channel may move at most 30% from its current spend (the slider allows 10% to 60%) and must stay inside the weekly range seen in the data. Here +2.6% is about $1,860 a week, roughly 1% of Brightwell's $190,000 weekly sales: enough to test in steps, not to reorganize the budget. Read it next to the binding-constraint column and the collinearity warning.

⚠️ Common Misread to Avoid: Booking the gain as guaranteed revenue. It is the model's prediction from its own fitted curves, limited to spend levels the brand has already tried. Shift in steps and confirm with a holdout or geo experiment before moving the whole amount.

📐 Technical Definition: The optimizer starts each channel at its lower bound. It hands out the rest of the budget in 500 small steps, each to the channel with the highest marginal ROI at that moment. That greedy rule ends where marginal returns are as equal as the bounds allow.

🎯 The Manager's Question: Of the customers who really said yes, what share did the model catch?

📊 Worked Business Example: At the 0.50 cutoff the influencer model flags 277 of the 463 posts that converted, a sensitivity of 59.8%. Lowering the cutoff to 0.30 catches 384, or 82.9%.

⚖️ How to Judge the Number: With no model, flagging a random 30% of customers catches 30% of the yes customers. Higher is better for the same number of contacts. Read it together with specificity: lowering the cutoff from 0.50 to 0.30 catches more yeses but drops specificity from 84.4% to 52.0% here.

⚠️ Common Misread to Avoid: Reading high sensitivity as a good model. It ignores false alarms entirely: a model that flags everyone scores 100% here and is useless. Read it with specificity or precision.

📐 Technical Definition: It is the vertical axis of the ROC curve and the recall in precision-recall terms. It cannot be computed without actual yeses.

🎯 The Manager's Question: Which customer group is each category's coefficient being compared with?

📊 Worked Business Example: With Mega as the reference, Micro's odds ratio of 1.74 compares Micro and Mega posts alike in views, swipe-ups and region. Raw Micro posts convert less often (34% against 73%), but at equal engagement the gap vanishes: its interval, 0.75 to 4.0, includes 1.

⚖️ How to Judge the Number: Each level other than the reference gets a 0/1 indicator, and its coefficient is the gap from the reference, which sits at 0 (an odds ratio of 1). Choose a reference that is large and meaningful, such as the level with the most rows (Micro, 478 posts) or current practice. Read each level's odds ratio against that reference, and its interval against 1.

⚠️ Common Misread to Avoid: Reading a negative category coefficient as a negative chance. It means lower log-odds than the reference group with the other predictors held fixed, so Mid at -0.09 converts slightly less often than Mega, though its interval spans 0. Changing the reference changes every category coefficient but none of the predicted probabilities.

📐 Technical Definition: For a predictor with k levels, k - 1 indicators enter beside the intercept, and all of them are 0 for the reference. The tool uses the level chosen in the reference drop-down, or the first level it finds.

🎯 The Manager's Question: Which category do all the other categories get compared against?

📊 Worked Business Example: The regional store preset uses Midwest as the baseline, so West shows as +13.9 and South as -9.1. Switch the baseline to South and West becomes +23.1, while every fitted value stays the same.

⚖️ How to Judge the Number: Pick the baseline that makes the gaps easiest to read, such as the largest group or the status quo. A baseline with fewer than about 5 rows makes every gap wobbly, because each one is measured against it. Read the category coefficients together with their standard errors after any switch.

⚠️ Common Misread to Avoid: Thinking a new baseline changes the model. The fit, the residuals and every diagnostic stay identical. Only the intercept and the category coefficients are re-expressed against the new baseline.

📐 Technical Definition: That simple form holds when the factor is the only predictor; with other predictors \(b_L\) is the same gap adjusted for them. Switching the baseline leaves the columns of the design matrix spanning the same space, so the fitted values and residuals are unchanged.

🎯 The Manager's Question: How far has the price shoppers think is fair slid below the $50 list price?

📊 Worked Business Example: At the NordicThreads defaults (40% off, a $30 deal price), the average fair price starts at $45.06. By the time the run settles at Year 18 it has drifted to $31.74, close to the deal price.

⚖️ How to Judge the Number: The anchor is the $50 list price. A shopper whose fair price falls below $45, 90% of list, starts waiting for a sale. Each season a shopper moves 15% of the way toward the average price that buying neighbors paid, or 25% once a competitor is matching prices. Read it next to Realized Gross Margin, where a falling fair price shows up as margin lost on every sale.

⚠️ Common Misread to Avoid: Believing a promo code only affects the shoppers who redeem it. Every shopper who sees neighbors pay $30 updates what $50 feels like. That is how full-price shoppers become sale-waiters without ever using a code.

📐 Technical Definition: Neighbors who are waiting for a sale buy nothing and are left out of the average. Shoppers start at $50 if they pay full price and at the deal price if they buy on deal. With a competitor matching prices, 0.85 and 0.15 become 0.75 and 0.25.

🎯 The Manager's Question: What value of {outcome} does the model predict with every numeric predictor at zero and every category at baseline?

📊 Worked Business Example: For the ad campaigns the intercept is $105.8k, the revenue the line predicts for a campaign with no spend. No campaign spent under $6.6k, so treat it as an anchor for the line, not a forecast.

⚖️ How to Judge the Number: It means something only when 0 on every numeric predictor lies inside the data. Read it next to the smallest predictor values: if no case is near zero, it is an extrapolation that just anchors the line. In a model with categories, it is the prediction for the baseline category.

⚠️ Common Misread to Avoid: Reading it as the starting or lowest value of {outcome}. It is often a point no case comes near, so an odd value, even a negative one, is not a sign the model is wrong. Centring the predictor moves it to the average case without changing the fit.

📐 Technical Definition: That is the one-predictor form. With several predictors it is the first element of \(\mathbf{b}\), where \(\mathbf{b}\) is the vector of estimated coefficients. It equals the fitted value for a case at 0 on every numeric predictor and in the baseline level of every factor.

🎯 The Manager's Question: How much higher is {outcome}, on average, where {predictor} is one unit higher and the other predictors match?

📊 Worked Business Example: For the 60 ad campaigns the slope is 0.665: each extra $1k of spend goes with about $665 more revenue on average. Because the true relationship bends, that one number understates the return at small budgets and overstates it at large ones.

⚖️ How to Judge the Number: 0 means no association once the other predictors are held fixed. Read the size in units of {outcome} per unit of {predictor}, next to its standard error and interval, which say how much it would wobble in a new sample. For the ad campaigns, $10k more spend goes with about $6.65k more revenue. Use it for a budget only after the linearity check passes.

⚠️ Common Misread to Avoid: Reading it as the effect of raising {predictor} by one unit. It is an association in this data, not the effect of an intervention: other things that moved with {predictor} are mixed in. Only an experiment or a strong design turns it into a causal effect.

📐 Technical Definition: That is the one-predictor form. With several predictors the tool solves \(\mathbf{b} = (\mathbf{X}^\top\mathbf{X})^{-1}\mathbf{X}^\top\mathbf{y}\), where \(\mathbf{X}\) is the design matrix and \(\mathbf{b}\) the vector of estimated coefficients. Its standard errors assume independent errors with equal spread.

🎯 The Manager's Question: How many conversions did this approach give up while it was finding the answer?

📊 Worked Business Example: In the default Nordvik Travel race (20,000 visitors, seed 42), the oracle earns 832 conversions. The A/B test earns 788, a regret of 44, and Thompson sampling earns 818, a regret of 14.

⚖️ How to Judge the Number: 0 means matching the oracle, which happens only by luck or when every option is equal. Judge it against the other contestants on the same visitors, and next to what each one learned. With the Monte Carlo view set to 500 races, the A/B test averages 71 conversions of regret to Thompson sampling's 18. Yet its interval on the losing hero is about half as wide, 1.1 against 2.0 points on average.

⚠️ Common Misread to Avoid: Reading regret as a mistake. An A/B test pays it on purpose, splitting traffic evenly to buy an estimate it can defend afterwards. The question is whether what you learned was worth the 44 conversions it cost.

📐 Technical Definition: The lab can count it only because it knows the true rates; a live test never can. In expectation it is \(\sum_t (p_{b(t)} - p_{a(t)})\): the true rate of the best option minus the rate of the one served, summed over visitors. Every contestant faces the same visitor draws, so it can never come out negative.

🎯 The Manager's Question: How many times as common is {outcome} in {groupA} as in {groupB}?

📊 Worked Business Example: In the starting table 120 of 1,000 people in Segment A convert (12%) against 40 of 1,000 in Segment B (4%). The risk ratio is 0.12 / 0.04 = 3.0, so Segment A converts three times as often.

⚖️ How to Judge the Number: 1 means {groupA} and {groupB} have the same rate. 2 means the rate in {groupA} is twice the rate in {groupB}, and 0.5 means half. Read it next to the two rates themselves, because the ratio hides the scale: 3 times a 4% rate is 8 more customers per 100, but 3 times a 0.1% rate is 0.2 more. That gap per 100 is what a budget decision needs.

⚠️ Common Misread to Avoid: Treating the risk ratio and the odds ratio as the same number. With 12% against 4% they are close (3.0 against 3.27) because both rates are low. With 60% against 30% the risk ratio is 2.0 but the odds ratio is 3.5, so quoting the odds ratio as "3.5 times as likely" overstates the gap.

📐 Technical Definition: From the 2x2 table, \(P_A = a/(a+b)\) and \(P_B = c/(c+d)\): each group is divided by its own total. Odds-O-Matic divides the two rates directly, with no model behind it. When both rates are below about 10%, \(\text{RR} \approx \text{OR}\); as the rates rise, the odds ratio moves further from 1 than the risk ratio does. The risk ratio can never exceed \(1/P_B\).

🎯 The Manager's Question: How many people would this portfolio miss if this item were dropped?

📊 Worked Business Example: Dropping Chocolate Fudge Brownie from Frost & Fold's best three costs 24.4 points of reach; dropping Strawberries & Cream costs 9.1. Those are the shoppers who like that flavor and neither of the other two.

⚖️ How to Judge the Number: 0 means every fan of the item is covered by another item in the set. Larger means more respondents rely on that item alone. Read it next to the Shapley credit and the best replacement: a small loss with a near-free swap means the slot could be spent elsewhere.

⚠️ Common Misread to Avoid: Reading a removal loss of zero as a worthless item. An item can share all its fans with others and still earn Shapley credit; it is simply not indispensable. Removal loss asks who you would lose, and credit asks who the item serves.

📐 Technical Definition: It equals the share of respondents whose only accepted item in the set is j, so the removal losses can never add up to more than the reach. Weighted data uses survey weights.

🎯 The Manager's Question: What is the lowest offer the publisher will take before leaving the ad space empty?

📊 Worked Business Example: The top banner's floor is $2.50 CPM, so FinBank's $1.80 bid is ineligible. Set the floor multiplier to 2 and the floor becomes $5.00: every bid fails and the space goes unsold.

⚖️ How to Judge the Number: A floor of $0 accepts any bid, and each step up removes the lowest offers first. Read it against the bids: under first price a floor below the top bid changes neither the winner nor the price, because the winner pays its own bid. A floor above the top bid leaves the space unsold, so watch the share of spaces that still sell.

⚠️ Common Misread to Avoid: Reading the floor as the price the winner pays. Under the first-price rule the winner pays its own bid: with a $2.50 floor, LuxeDerma still pays $4.20.

📐 Technical Definition: In the teaching auction a bid is eligible when its CPM is at least the floor CPM. The floor multiplier scales all three spaces (top banner $2.50, article $2.00, sidebar $3.00). Real platforms also apply fees, policy and timing rules before the comparison.

🎯 The Manager's Question: Is there a bend in the relationship that a straight-line model is missing?

📊 Worked Business Example: For the 60 ad campaigns RESET gives F = 87.4 with p below 0.001. If the relationship were truly straight, a bend this clear would almost never turn up.

⚖️ How to Judge the Number: A p-value below {alpha} flags a bend. Read it next to the residuals-against-fitted plot to see its shape, and the squared-term test for each predictor to see where it is. A bend means the slope averages different rates, so model the curve before you budget off it.

⚠️ Common Misread to Avoid: Reading a pass as proof the relationship is straight. Small samples can miss a real bend. A high R² says nothing either way: the ad campaigns have an R² of 0.87 and still fail RESET.

📐 Technical Definition: The tool tests \(\gamma = 0\) with a t-test on \(n-k-1\) degrees of freedom, where n is the number of cases and k the number of coefficients in the original model. Its F is that t squared, matching statsmodels linear_reset with power 2. Some packages also add a cubed term.

🎯 The Manager's Question: For this case, how far did the actual value of {outcome} land above or below the prediction?

📊 Worked Business Example: The $6.6k campaign was predicted to bring $110.1k in revenue and brought $88.5k. Its residual is 88.5 minus 110.1, or -21.6: a $21.6k shortfall.

⚖️ How to Judge the Number: Positive means the case beat the prediction, negative means it fell short, and 0 is a perfect hit. It is in the units of {outcome}, so read it against the typical miss, the residual standard error. For the ad campaigns that is $7.26k, so a $21.6k miss is about three typical misses.

⚠️ Common Misread to Avoid: Treating a big residual as a reason to delete the case. It is a case the model does not explain yet. Find out why, a data error or a real difference, before you drop anything.

📐 Technical Definition: With an intercept in the model, least squares makes the residuals sum to zero and leaves them uncorrelated with every predictor. So a pattern in a residual plot is something the model has missed, not something it put there.

🎯 The Manager's Question: Does the model miss by more in some categories than in others?

📊 Worked Business Example: In the B2B deals-by-industry preset, Software deals miss by about $25.4k while Manufacturing deals miss by about $6.6k, a ratio of 3.9. One pooled standard error understates the uncertainty for Software and overstates it for Manufacturing.

⚖️ How to Judge the Number: Equal spread gives a ratio of 1. The tool compares the widest category with the narrowest, using only categories with at least 5 rows, and treats a ratio of 2 or more as wide. Read it next to the Breusch-Pagan p-value, and report robust standard errors when both point the same way.

⚠️ Common Misread to Avoid: Mixing up spread with average. This compares how widely each category misses, not whether it misses high or low on average. Two categories can both average a zero miss and still differ fourfold in spread.

📐 Technical Definition: The ratio reported is \(\max_L s_L / \min_L s_L\), the widest level's spread over the narrowest. Levels with fewer than 5 rows are left out because their spread is too unstable to compare.

🎯 The Manager's Question: How far off is a typical prediction, in units of {outcome}?

📊 Worked Business Example: For the 60 ad campaigns the residual standard error is $7.26k. If the misses were bell-shaped, about two thirds of campaigns would land within $7.26k of the line and about 95% within $14.5k.

⚖️ How to Judge the Number: Zero would mean every case sits on the line. Read it against the typical size of {outcome}: a $7.26k miss on campaigns averaging $141k in revenue is about 5%. Twice it gives a rough 95% band for a single case, if the misses are bell-shaped.

⚠️ Common Misread to Avoid: Mixing it up with the standard error of a coefficient. This one describes how far individual cases miss the line, not how much the slope would wobble in a new sample. The slope's standard error shrinks as rows are added; this one does not.

📐 Technical Definition: Dividing by \(n - k\) rather than by the number of cases allows for the coefficients the fit used up. Its square is statsmodels' mse_resid, and R prints it as "Residual standard error".

🎯 The Manager's Question: Does a straight line fit the whole range of predictions, or does the pattern bend?

📊 Worked Business Example: In the 60 ad campaigns, the misses run below zero at small and large budgets and above zero in the middle, an arch. Spend from $7k to $100k does not buy revenue at a steady rate.

⚖️ How to Judge the Number: A good plot is an even band around the 0 line with a flat smoother. An arch or a U is a bend; read it next to the RESET test, which puts a p-value on it (below 0.001 for the ad campaigns). A bend means the slope is an average of different rates, so do not budget off it.

⚠️ Common Misread to Avoid: Reading an arch as a few outliers. When the misses curve as a group, the relationship itself is not straight, as with diminishing returns to ad spend. Removing the cases at the ends would hide the bend, not fix it.

📐 Technical Definition: Each case is plotted at its fitted value along the bottom and its residual up the side. The tool draws a LOWESS smoother through the points to make a bend easier to see. With several predictors, a bend here says some predictor is curved; the per-predictor plots and squared-term tests say which.

🎯 The Manager's Question: Does the model miss in a pattern, or only at random?

📊 Worked Business Example: At Wrenfield the network’s 500 held-back markets form a level band: the typical miss is about $12,300 where it predicts under $129,000 and $12,900 where it predicts over $155,000. So one RMSE, $12,400, fits small and large markets alike.

⚖️ How to Judge the Number: A level band centered on 0 is the target: misses above and below balance at every prediction. Compare the spread at the left and right ends; at Wrenfield both are about $12,000. A fan calls for logging the outcome before trusting one RMSE or the ranges; a curve calls for a squared term in the regression. Read it next to the RMSE, one average miss for every row, which hides a fan.

⚠️ Common Misread to Avoid: Calling a widening fan random noise. A fan means the misses get bigger for bigger predictions: a model that misses by about $5,000 on small markets may miss by $25,000 on large ones, and one RMSE averages the two.

📐 Technical Definition: Each held-back row’s residual is plotted against its predicted value, in the outcome’s own units. A positive residual means the model predicted too low.

🎯 The Manager's Question: Is subscription revenue now above or below where it started in Year 0?

📊 Worked Business Example: Sanjay expects about +8.6% from an 18% increase that loses 8% of teams. In the default run the index jumps to 107.6 at the announcement, then settles at 105.9 by Year 14 as neighbors follow the leavers out.

⚖️ How to Judge the Number: 100 means the same revenue as Year 0. Above 100 the higher price more than pays for the lost teams; below 100 it does not. Read it next to the no-shock run on the same card, which sits at 99.9 at the defaults, and next to Active Subscriber Share.

⚠️ Common Misread to Avoid: Judging the increase by the index on announcement day. At 18% it starts at 107.6 and ends at 105.9; at 21% it also starts above 100 but ends at 0 once every team has left. Watch where the index ends, not where it starts.

📐 Technical Definition: Every team pays the same price, so revenue is active teams times the price multiplier. Cost is not modelled, so the index tracks revenue, not profit.

🎯 The Manager's Question: Which item of the pair should carry the cross-sell prompt for the other?

📊 Worked Business Example: In Corner Market, 58.9% of pasta baskets hold pasta sauce, but 92.9% of pasta-sauce baskets hold pasta. The lift is 3.37 both ways; the gap only reflects that pasta is in 1,378 baskets and sauce in 873.

⚖️ How to Judge the Number: Compare the two confidences; equal values mean the two items are equally common. A gap like 92.9% against 58.9% says sauce buyers nearly always have pasta already, while 41% of pasta baskets leave without sauce. So the prompt belongs on pasta, suggesting sauce. Read both next to the shared lift, which is the same either way.

⚠️ Common Misread to Avoid: The 92.9% direction, sauce to pasta, is read as the stronger rule to act on. It is the direction where a prompt is nearly redundant: only 62 sauce baskets lack pasta, against 567 pasta baskets that lack sauce. Both directions share one lift; the confidence gap only says where the missing item is.

📐 Technical Definition: The tool prints the reverse confidence in the rule detail, rounded to a whole percent: 93% for sauce to pasta. The asymmetry comes purely from how common the two items are, so it says nothing about which purchase leads.

🎯 The Manager's Question: How many times as common is {outcome} in {groupA} as in {groupB}?

📊 Worked Business Example: In the starting table 120 of 1,000 people in Segment A convert (12%) against 40 of 1,000 in Segment B (4%). The risk ratio is 0.12 / 0.04 = 3.0, so Segment A converts three times as often.

⚖️ How to Judge the Number: 1 means {groupA} and {groupB} have the same rate. 2 means the rate in {groupA} is twice the rate in {groupB}, and 0.5 means half. Read it next to the two rates themselves, because the ratio hides the scale: 3 times a 4% rate is 8 more customers per 100, but 3 times a 0.1% rate is 0.2 more. That gap per 100 is what a budget decision needs.

⚠️ Common Misread to Avoid: Treating the risk ratio and the odds ratio as the same number. With 12% against 4% they are close (3.0 against 3.27) because both rates are low. With 60% against 30% the risk ratio is 2.0 but the odds ratio is 3.5, so quoting the odds ratio as "3.5 times as likely" overstates the gap.

📐 Technical Definition: From the 2x2 table, \(P_A = a/(a+b)\) and \(P_B = c/(c+d)\): each group is divided by its own total. Odds-O-Matic divides the two rates directly, with no model behind it. When both rates are below about 10%, \(\text{RR} \approx \text{OR}\); as the rates rise, the odds ratio moves further from 1 than the risk ratio does. The risk ratio can never exceed \(1/P_B\).

🎯 The Manager's Question: By how much are the predictions typically off, in {outcome}'s own units?

📊 Worked Business Example: In the EDM event-spend preset patrons spend $57.10 on average with an SD of $16.68, and the model's RMSE is about $11.0. That beats guessing the average, whose RMSE would be the SD, by about a third.

⚖️ How to Judge the Number: The no-model figure is the SD of {outcome}: guessing the average for everyone has an RMSE equal to it. Lower is better. Read the holdout figure against the training figure: under 10% higher is fine, 10% to 25% worth watching, and more means the model memorised.

⚠️ Common Misread to Avoid: Reading RMSE as the average miss. Squaring counts big misses extra, so a few bad predictions inflate it. MAE is the average miss; here it is $8.8 against an RMSE of $11.0.

📐 Technical Definition: For a transformed outcome the tool converts predictions back first, so the figure is in real units. It divides by n, not by the residual degrees of freedom, so it sits slightly below the residual standard error.

🎯 The Manager's Question: Within a fixed window, how many of its months does the average customer stay active?

📊 Worked Business Example: Over Parcel & Pantry's 18-month horizon the average customer stays 9.4 months, about half the window. Promotion sign-ups stay 5.4 months; every other channel stays about 11.

⚖️ How to Judge the Number: The anchor is the horizon itself: 18 of 18 months would mean nobody left, and 9.4 means half the window is used. Compare groups only at the same horizon, because a longer window always gives a bigger number. Read it next to the median, which can be missing when this number cannot.

⚠️ Common Misread to Avoid: Reading it as the average customer lifetime. It stops counting at the horizon, so a customer still active at month 18 adds 18, not their eventual total. Two groups can share one value while losing customers at quite different ages inside the window.

📐 Technical Definition: The tool adds up the rectangles under the step curve between event times, which is the exact area under a step function. It is always defined, unlike the median, which is why it is the number to quote when the curve stays above 50%.

🎯 The Manager's Question: Did this channel bring back more revenue than it cost, and by how much?

📊 Worked Business Example: In the lab, email spent $2,400 and produced $49,995, a ROAS of 20.8x. Paid social spent $18,500 and produced $14,299, a ROAS of 0.77x: every dollar came back as 77 cents.

⚖️ How to Judge the Number: 1.0x is the anchor: revenue equal to spend. A channel pays for itself only above 1 divided by the gross margin, so 1.82x at GlowRoute's 55% margin, the same bar the Web Analytics Sandbox uses. Read it next to the spend it is built on, since a 20x return on $2,400 says nothing about the twenty-first dollar.

⚠️ Common Misread to Avoid: Reading a 20x ROAS as "put the whole budget here". The ratio describes the dollars already spent at this scale. Email's list is the same size tomorrow, so the next $30,000 cannot return what the first $2,400 did.

📐 Technical Definition: The lab's "true ROAS" uses revenue from the ground-truth session log, not the dashboard, so a misfiled channel still shows its real return after the reveal. Revenue is sales dollars, not profit.

🎯 The Manager's Question: How many more real yes customers do we catch as we accept more false alarms?

📊 Worked Business Example: At the 0.50 cutoff the influencer model sits at 15.6% false alarms and 59.8% caught. Lowering the cutoff to 0.30 moves the dot to 48.0% false alarms and 82.9% caught.

⚖️ How to Judge the Number: The dashed diagonal is a random ranking: flagging 20% of each group lands at (20%, 20%). Points toward the top-left are better trade-offs, and the area under the curve sums it up from 0.5 to 1. Read it with the AUC and with the cost of each kind of error, which picks the dot.

⚠️ Common Misread to Avoid: Reading the axes as predicted probability or percent correct. Each dot is one cutoff applied to the same scores, and the two axes divide by different groups: false alarms among actual noes, catches among actual yeses. Moving the cutoff moves the dot, not the model or its AUC.

📐 Technical Definition: For each cutoff c the tool flags rows with a predicted probability of at least c and plots FPR across and TPR up. Both outcomes must be present. The curve uses the fitting rows, and Validation & scoring reports the AUC on held-back rows. It measures ranking, not calibration.

🎯 The Manager's Question: How much does the publisher earn from ads for every 1,000 page visits?

📊 Worked Business Example: An advertiser pays $5 CPM and the selling platform keeps a 20% fee, so the publisher nets $4 per 1,000 billable impressions. If all three spaces on the page sell at that price, page RPM is 3 × $4 = $12.

⚖️ How to Judge the Number: It is $0 when no space sells, and it rises with the price, the share of spaces filled and the number of spaces per page. Compare it only against figures on the same revenue basis, before or after fees. Read it next to the reader experience: a fourth ad space can lift it while driving readers away.

⚠️ Common Misread to Avoid: Reading page RPM as the CPM an advertiser pays. A $5 CPM to the buyer is $4 to the publisher after a 20% fee, and page RPM then adds up every ad space on the page.

📐 Technical Definition: The 20% fee is the simulator's illustrative assumption, not a market average. Its campaign model leaves out the publisher's other advertisers, unsold spaces and costs.

🎯 The Manager's Question: How far off is a typical prediction, in units of {outcome}?

📊 Worked Business Example: For the 60 ad campaigns the residual standard error is $7.26k. If the misses were bell-shaped, about two thirds of campaigns would land within $7.26k of the line and about 95% within $14.5k.

⚖️ How to Judge the Number: Zero would mean every case sits on the line. Read it against the typical size of {outcome}: a $7.26k miss on campaigns averaging $141k in revenue is about 5%. Twice it gives a rough 95% band for a single case, if the misses are bell-shaped.

⚠️ Common Misread to Avoid: Mixing it up with the standard error of a coefficient. This one describes how far individual cases miss the line, not how much the slope would wobble in a new sample. The slope's standard error shrinks as rows are added; this one does not.

📐 Technical Definition: Dividing by \(n - k\) rather than by the number of cases allows for the coefficients the fit used up. Its square is statsmodels' mse_resid, and R prints it as "Residual standard error".

🎯 The Manager's Question: How many extra baskets hold this pair, beyond what coincidence alone would produce?

📊 Worked Business Example: In Corner Market, 811 baskets hold pasta and pasta sauce where chance predicts 241: a leverage of 0.114, or about 570 extra baskets in 5,000. Truffle oil with burrata, despite a lift of 37.6, adds about 8.

⚖️ How to Judge the Number: Zero means exactly as often as chance; negative means less often. Multiply by the number of baskets to get extra baskets: 0.02 on 5,000 baskets is 100 more than chance, a quantity you can put a margin against. Read it next to rule lift: lift says how strong, leverage says how much volume, and an end cap needs both.

⚠️ Common Misread to Avoid: Leverage looks tiny beside lift, so it is read as unimportant. It is the one measure here that rarity cannot inflate: 0.114 for pasta and sauce against 0.0016 for truffle oil and burrata is the real gap in business between them.

📐 Technical Definition: Unlike lift it is bounded, between -0.25 and 0.25, so it is the steadier measure for ranking when counts vary widely. The tool computes it for every rule and puts it in the exported rules file, not in the on-screen table.

🎯 The Manager's Question: How many times as often are these bought together as coincidence would predict?

📊 Worked Business Example: Pasta and pasta sauce share a basket 811 times in Corner Market, where coincidence predicts about 241. That is a rule lift of 3.37: they travel together 3.4 times as often as chance.

⚖️ How to Judge the Number: 1.0 is exactly coincidence; below 1 the items avoid each other, which is a finding of its own. Above 1 it is a multiplier: 3.0 means three times as often as chance. Read it together with the basket count and the q-value, because a rare pair can show a huge lift on a handful of baskets. A lift above 2 on hundreds of baskets is worth a shelf test.

⚠️ Common Misread to Avoid: A lift of 37.6 is read as the best rule in the file. With the minimum lowered to 5, truffle oil and burrata reach it on just 8 baskets, because lift flatters rare items the way confidence flatters popular ones. It is a real pattern and far too thin to plan a shelf around.

📐 Technical Definition: Equivalently \(P(X \text{ and } Y) / (P(X)P(Y))\), so a rule and its reverse share one lift though their confidences differ. It is not the promotional lift of a campaign, which compares sales with a baseline over time. At the default minimum of 20 the tool tags a rule "Too rare to act on" when its lift is 5 or more on fewer than 40 baskets.

🎯 The Manager's Question: Could a pattern this strong be luck, given how many rules were tested at once?

📊 Worked Business Example: At the default settings Corner Market tests 16,991 rules, and 242 survive the correction. Beer and diapers, the famous folklore pair, has q = 0.74: 29 shared baskets is close to the 24 that chance would give.

⚖️ How to Judge the Number: Judge it against 0.05, as you would a p-value, but the promise differs. Accept every rule with q below 0.05 and about 5% of the accepted rules are expected to be false, not a 5% chance of any false rule at all. Read it next to the basket count and rule lift, because a tiny q on 8 baskets is a real but useless pattern.

⚠️ Common Misread to Avoid: A small q is read as a rule worth acting on. It only says the pattern is probably not luck. Truffle oil and burrata have q below 0.001 on 8 baskets and are still not worth moving a shelf for.

📐 Technical Definition: The raw p-value is a one-sided Fisher exact test that the pair turns up together more often than chance. The tool applies Benjamini-Hochberg across every rule evaluated, before any display filter, because correcting after filtering would defeat the purpose. It controls the false discovery rate, not the chance of any single false rule.

🎯 The Manager's Question: Does a straight line fit the whole range of predictions, or does the pattern bend?

📊 Worked Business Example: In the 60 ad campaigns, the misses run below zero at small and large budgets and above zero in the middle, an arch. Spend from $7k to $100k does not buy revenue at a steady rate.

⚖️ How to Judge the Number: A good plot is an even band around the 0 line with a flat smoother. An arch or a U is a bend; read it next to the RESET test, which puts a p-value on it (below 0.001 for the ad campaigns). A bend means the slope is an average of different rates, so do not budget off it.

⚠️ Common Misread to Avoid: Reading an arch as a few outliers. When the misses curve as a group, the relationship itself is not straight, as with diminishing returns to ad spend. Removing the cases at the ends would hide the bend, not fix it.

📐 Technical Definition: Each case is plotted at its fitted value along the bottom and its residual up the side. The tool draws a LOWESS smoother through the points to make a bend easier to see. With several predictors, a bend here says some predictor is curved; the per-predictor plots and squared-term tests say which.

🎯 The Manager's Question: What does each customer bring in on average, the figure that scales up to a total?

📊 Worked Business Example: Flow A's 95 orders average $113.38, but without its single $4,120 bulk order the other 94 average $70.76. One case moves the average by more than $40.

⚖️ How to Judge the Number: It is the balance point, where the deviations above and below sum to 0. Read it next to the median: a mean a quarter or more above the median (1.25 times), like Flow A's $113 against $63, is a few big values pulling it up. It is the right number for totals and budgets, since total revenue is the mean times the count.

⚠️ Common Misread to Avoid: Reading the mean as what a typical customer delivers. In skewed data such as customer lifetime value, most cases fall below it: 84 of Flow A's 95 orders sit under its $113 mean. Yet for untargeted acquisition the mean is the expected value E[X] per customer you sign. So capping acquisition cost (CAC) at the median rather than the mean leaves profitable expected return on the table.

📐 Technical Definition: For 0/1 data, such as whether a click converted, \(\bar{x} = \frac{1}{n}\sum x_i\) is the conversion rate: Aura's 70 sales in 1,850 clicks average 0.038. The mean has no resistance to outliers, so one arbitrarily large value moves it arbitrarily far.

🎯 The Manager's Question: How far does a typical row (customer, store, week) sit from the average on this measure?

📊 Worked Business Example: In the influencer preset swipe-ups average 933 per post with an SD of 1,019. An SD larger than the mean warns of a long right tail: the median post has 614 while one reaches 9,421.

⚖️ How to Judge the Number: It is in the measure's own units, and 0 means every row has the same value. For a roughly bell-shaped measure about two thirds of rows sit within one SD of the mean. Read it next to the mean and the median. For a measure that cannot go below 0, such as spend, counts or views, an SD near or above the mean, or a median well below it, signals a long right tail. Then summarise with the median, or consider a log transform before modelling.

⚠️ Common Misread to Avoid: Reading the SD as the uncertainty in the average. It describes how much individual posts differ; the standard error of the mean is the SD divided by the square root of n, here 1,019 over 31.6, about 32. The two answer different questions.

📐 Technical Definition: This is the sample formula, dividing by n - 1 as Excel's STDEV.S and pandas do, so it needs at least two values. A population SD divides by n and comes out slightly smaller.

🎯 The Manager's Question: How many cases is this answer built on?

📊 Worked Business Example: Aura's test rests on 1,850 clicks. Big Data Mode multiplies that by 100 at the same 3.8% rate, and the standard error falls only 10-fold, from 0.44 to 0.044 points.

⚖️ How to Judge the Number: No fixed number counts as enough; what you need depends on the smallest gap worth detecting. Read it next to the interval width: 4 times the cases halves it. With rare conversions, precision tracks the number of conversions (Aura's 70) more than the number of clicks.

⚠️ Common Misread to Avoid: Reading a big n as a safe answer. If the 1,850 clicks came only from Aura's most loyal followers, 18,500 more of the same would make the estimate more precisely wrong. Size fixes noise, not who was sampled.

📐 Technical Definition: The number of independent observations the test used, after missing values were dropped; the one-sample tool shows it as Active Sample (N). Repeated measures on the same person or store are not independent, and counting them separately overstates n.

🎯 The Manager's Question: At what weekly spend does this channel reach half of the most it could ever deliver?

📊 Worked Business Example: Brightwell's TV reaches half its maximum at about $24,000 a week (the slider's K of $60,000 × (1 - 0.6)) and averages $13,600: still on the steep part. Search reaches half at about $2,000 a week but averages $6,000, three times past it.

⚖️ How to Judge the Number: At a steady weekly spend of K × (1 - λ) a channel delivers 50% of its maximum, and the next dollar earns a quarter of what the first did. Below that spend there is headroom; at two or three times it most of the effect is used up. Compare that figure, not K itself, with the channel's average weekly spend, then confirm with marginal ROI.

⚠️ Common Misread to Avoid: Comparing K straight with weekly spend. K is measured in carried-over spend, which builds up over the weeks, so at a steady weekly spend half saturation arrives at K × (1 - λ). For TV that is $60,000 × 0.4 = $24,000 a week, not $60,000.

📐 Technical Definition: The tool uses the Hill curve with shape 1, which bends from the first dollar and has no S-shaped threshold. K is in adstocked units to match the fit; at a steady weekly spend x the carried-over level is \(x/(1 - \lambda)\), so half saturation comes at \(x = K(1 - \lambda)\). Response past the highest observed week is extrapolation, drawn dashed on the tool's curves.

🎯 The Manager's Question: Do the misses stay the same size as predictions grow, or do they balloon at larger values?

📊 Worked Business Example: In the B2B deal-size preset, the residual spread is about $12k for the smallest third of deals and $29k for the largest third, a ratio of 2.5. The Scale-Location smoother climbs from left to right.

⚖️ How to Judge the Number: Look for a flat smoother through an evenly thick cloud. A rising smoother means misses grow with the prediction; the tool also compares the residual spread in thirds of the fitted values and treats a ratio of 2 or more as wide. Read it alongside the Breusch-Pagan test, which puts a p-value on the trend.

⚠️ Common Misread to Avoid: Thinking uneven spread biases the slope. The fitted line stays centred; what breaks are the standard errors, so the p-values and intervals are off. Robust standard errors or a log of the outcome repair that.

📐 Technical Definition: The tool plots \(z_i\) against the fitted values, where \(r_i = e_i/(s\sqrt{1-h_{ii}})\) is the standardized residual, \(e_i\) the residual, \(s\) the residual standard error and \(h_{ii}\) the leverage. The square root tames the skew of absolute values, so a trend in spread shows as a tilt in the smoother.

🎯 The Manager's Question: What single score should each respondent get on each factor for reporting and later analysis?

📊 Worked Business Example: A NovaBrew respondent who answers 6, 7 and 5 on the three Quality items gets a Quality score of 6.0, on the survey's own 1 to 7 scale. A manager can read 6.0 as "agree" without a lesson on z-scores.

⚖️ How to Judge the Number: The score keeps the survey's own scale, so the midpoint is the neutral anchor: 3 on a 1 to 5 scale, 4 on 1 to 7. Read it next to the factor's alpha or omega. Report a score only for a factor at 0.70 or above, since averaging items that do not agree averages noise.

⚠️ Common Misread to Avoid: Thinking the simple mean throws the factor analysis away. On a clean scale, weighted factor scores and the simple mean rank respondents almost identically, and the mean keeps units a manager can read. The analysis still decides which items go into each mean.

📐 Technical Definition: The tool averages each respondent's answered items on the factor, so a skipped item drops out of that person's mean rather than counting as 0. Items are assigned by their largest pattern loading, and only items still ticked count. Regression or Bartlett factor scores are not offered, by design.

🎯 The Manager's Question: On average, how many of a household's next-door neighbors belong to its own group?

📊 Worked Business Example: Brookhaven starts at 33.7%, close to the 33% that random placement of three equal groups gives. The default run settles at Year 21 at 71.7%, and 77% of households then have a same-group majority next door.

⚖️ How to Judge the Number: Random placement of three equal groups gives about 33%, and 100% means every occupied neighbor matches. Compare the end value against τ: the distance between them is the Emergence Gap, sorting beyond what anyone required. Read it next to Moran's I, which says whether like households form blocks, and the SD across households, which says whether the sorting is even or patchy.

⚠️ Common Misread to Avoid: Assuming a town this sorted must be full of households who want one-group streets. Every household here asks for only 33% and the town still reaches 71.7%. Mild preferences, acting on each other, are enough.

📐 Technical Definition: Neighbors are the lots within the radius slider (8 at r = 1) on a grid whose edges wrap. Vacant lots count as neither same nor different, and a household with no occupied neighbor is left out. The SD of \(m_i / o_i\) across households is reported beside the mean.

🎯 The Manager's Question: Did this row get a prediction, and if not, what stopped it?

📊 Worked Business Example: In a scored file of 500 rows, 497 read "scored" and 3 name a problem, such as a blank tenure or a plan the model never saw in training. Those 3 rows keep their place in the file, with blank prediction cells.

⚖️ How to Judge the Number: "scored" means the row had everything the model needs, and the target is 0 rows refused. Anything else names the problem, such as a blank cell or a category the model never saw in training. The segmentation tool is the exception: a respondent with some blank answers is still scored from the answers given, and only an all-blank row is refused. Read the count of refusals against the rows uploaded before trusting any summary of the scores.

⚠️ Common Misread to Avoid: An unscored row is not a prediction of zero or "no". Do not fill the blank in or quietly drop the row; fix the data, or retrain with that category included. A category the model never saw has no coefficient, so any number given to it would be invented.

📐 Technical Definition: Each row is checked before it is scored instead of being coerced. Required columns must be present and non-blank, and numbers are parsed the way the import step parsed the training data. Categories are matched against the training levels.

🎯 The Manager's Question: What did the model add to my file, and what can I safely do with it?

📊 Worked Business Example: Upload 500 new subscribers and the file comes back with your columns in their order, then the model’s columns on the right: a prediction column and score_status. All 500 rows are kept, including any that could not be scored, whose prediction cells stay blank.

⚖️ How to Judge the Number: Your own columns come back in the same order, and the model's columns are added on the right; a 500-row upload returns 500 rows. Every row gets a score_status, so count the "scored" rows against the rows you uploaded. Each added column has its own explanation in the preview's column headers. A column of your own with one of those names, such as score_status, is replaced. A blank prediction cell means the row was not scored, never zero or "no".

⚠️ Common Misread to Avoid: Reading the added columns as facts about these customers. They are estimates, only as good as the model's score on rows it never saw, and only for customers who resemble the training rows.

📐 Technical Definition: The fitted model is applied row by row with everything it learned in training: its coefficients or splits, its category levels and reference groups, and its means and scaling. Nothing is re-estimated from the uploaded file.

🎯 The Manager's Question: Where do extra factors stop adding real structure and start adding noise?

📊 Worked Business Example: In the default survey the eigenvalues fall from 2.85 to 1.05, then drift down slowly from 0.82 to 0.55. The elbow comes after the first factor, which agrees with parallel analysis.

⚖️ How to Judge the Number: Each point is one factor's eigenvalue, measured in items: 8 items give a total of 8. Look for the elbow where the steep drop turns into a flat tail, and keep the factors before it. Read it against the dotted parallel-analysis line on the same chart; a point above that line beats random data.

⚠️ Common Misread to Avoid: Treating the elbow as an exact cut-off. Two analysts can see it in different places, and a gentle curve may have no clear elbow at all. When it is unclear, use the parallel-analysis count, which the tool draws on the same chart.

📐 Technical Definition: The tool plots the eigenvalues of the full item correlation matrix from largest to smallest. It adds the parallel-analysis benchmark (dotted) and Kaiser's line at 1 (dashed). Because the eigenvalues add up to the number of items, an eigenvalue divided by p is that factor's share of the total: 2.85 of 8 is about 36%.

🎯 The Manager's Question: What percentage of your eligible survey completers are removed by your data-quality scrubbing rules?

📊 Worked Business Example: In Bramblecup Cold Brew, conservative scrubbing removes 44 careless responders out of 340 eligible completers. That yields a 12.9% Stage 2 scrubbing attrition rate.

⚖️ How to Judge the Number: 5% to 15% attrition is typical for campus and panel surveys, read alongside the Demographic Drift check and the 4-Tier Robustness Matrix.

⚠️ Common Misread to Avoid: Trying multiple custom attrition cutoffs until a hypothesis test crosses p < 0.05. Always report results across the pre-specified 4-Tier Robustness Matrix to prevent scrubbing-based p-hacking.

📐 Technical Definition: Ethical reporting separates Stage 1 administrative exclusions (previews and partial dropouts) from Stage 2 behavioral exclusions. Both counts belong in an APA CONSORT participant flow statement.

🎯 The Manager's Question: How far does a typical row (customer, store, week) sit from the average on this measure?

📊 Worked Business Example: In the influencer preset swipe-ups average 933 per post with an SD of 1,019. An SD larger than the mean warns of a long right tail: the median post has 614 while one reaches 9,421.

⚖️ How to Judge the Number: It is in the measure's own units, and 0 means every row has the same value. For a roughly bell-shaped measure about two thirds of rows sit within one SD of the mean. Read it next to the mean and the median. For a measure that cannot go below 0, such as spend, counts or views, an SD near or above the mean, or a median well below it, signals a long right tail. Then summarise with the median, or consider a log transform before modelling.

⚠️ Common Misread to Avoid: Reading the SD as the uncertainty in the average. It describes how much individual posts differ; the standard error of the mean is the SD divided by the square root of n, here 1,019 over 31.6, about 32. The two answer different questions.

📐 Technical Definition: This is the sample formula, dividing by n - 1 as Excel's STDEV.S and pandas do, so it needs at least two values. A population SD divides by n and comes out slightly smaller.

🎯 The Manager's Question: If I ran this again with a fresh sample, how much would this number move?

📊 Worked Business Example: Single Aura clicks either convert or not, so their standard deviation is about 19 percentage points. The standard error of the 3.8% rate across 1,850 clicks is only 0.44 points.

⚖️ How to Judge the Number: Read it against the gap it belongs to: the gap between the estimate and the no-change value (0 for a coefficient, the baseline for a rate). That gap divided by the standard error is roughly the z or t statistic. About 2 is the usual 0.05 line; Aura's 1.4-point lift is about 3 standard errors above 2.4%. It falls with the square root of the sample size, so 4 times the data halves it.

⚠️ Common Misread to Avoid: Reading the standard error as the spread of your customers. That is the standard deviation, about 19 points for Aura's single clicks. The standard error is how much the summary, the 3.8% rate, would wobble from one sample of 1,850 to the next: 0.44 points.

📐 Technical Definition: For a proportion \(s = \sqrt{\hat{p}(1-\hat{p})}\), so Aura's standard error is \(\sqrt{0.038 \times 0.962 / 1850} = 0.0044\). The one-sample z test divides by the baseline's \(\sqrt{p_0(1-p_0)/n} = 0.0036\) instead, because it asks how the data would look if 2.4% were true. Both assume independent cases, which repeat visits by one shopper break.

🎯 The Manager's Question: How sure can we be about which segment this respondent belongs to?

📊 Worked Business Example: In a 3-segment GreenLeaf solution, a respondent might score 0.81 for one segment, 0.15 for another and 0.04 for the third. They sum to 1, and at 0.81 the model is fairly sure but not certain.

⚖️ How to Judge the Number: The probabilities add to 1 across segments. If the answers told the model nothing, each would equal its segment's share of the sample. At or above 0.70 the tool calls the assignment clear; below that it flags the respondent as ambiguous. Read it next to the segment sizes, then decide who gets a segment-specific message and who gets the general one.

⚠️ Common Misread to Avoid: Reading 0.90 as "90% of their answers match this segment". It is not the share of the respondent's answers that match; it is how sure the model is that this segment, rather than another, produced them. So it says which segment fits best, not that any segment fits well.

📐 Technical Definition: This is Bayes' rule with the fitted segment sizes and profiles. Categorical answers use each segment's answer shares, and numeric answers use a normal curve on the study's own scaling. Blank answers are skipped, which pulls the probabilities toward the segment sizes.

🎯 The Manager's Question: Does the new file contain the same mix of segments as the original study?

📊 Worked Business Example: If a segment held 30% of GreenLeaf's 600 study respondents and 22% of a new file of 400, the shift is -8 points. With 400 new respondents, chance alone moves a 30% share by about 4.6 points, so this one is worth a look.

⚖️ How to Judge the Number: 0 points means the same mix. How much is noise depends on the file size: for a 30% segment, chance can move the share by about 9 points with 100 new respondents and about 3 points with 1,000. The tool notes shifts of 10 points or more; read each against the new file's size before re-planning segment budgets.

⚠️ Common Misread to Avoid: Treating a shift as proof that the market changed. A different recruitment channel, season or question wording can move the shares just as much. Check how the new file was collected first.

📐 Technical Definition: The chance band is about \(\pm 2\sqrt{s(1-s)/n}\), where s is the study share and n is the number of new respondents. Shares use modal assignments, so ambiguous respondents count toward their top segment.

🎯 The Manager's Question: Of the customers who really said yes, what share did the model catch?

📊 Worked Business Example: At the 0.50 cutoff the influencer model flags 277 of the 463 posts that converted, a sensitivity of 59.8%. Lowering the cutoff to 0.30 catches 384, or 82.9%.

⚖️ How to Judge the Number: With no model, flagging a random 30% of customers catches 30% of the yes customers. Higher is better for the same number of contacts. Read it together with specificity: lowering the cutoff from 0.50 to 0.30 catches more yeses but drops specificity from 84.4% to 52.0% here.

⚠️ Common Misread to Avoid: Reading high sensitivity as a good model. It ignores false alarms entirely: a model that flags everyone scores 100% here and is useless. Read it with specificity or precision.

📐 Technical Definition: It is the vertical axis of the ROC curve and the recall in precision-recall terms. It cannot be computed without actual yeses.

🎯 The Manager's Question: Does this text mix strong positive praise and negative complaints at the same time?

📊 Worked Business Example: A review with 0.30 positive token share and 0.30 negative token share scores 1.00 on ambivalence. A purely positive review scores 0.00.

⚖️ How to Judge the Number: Scores run from 0.00 (one-sided) to 1.00 (equally split), read alongside the VADER compound score.

⚠️ Common Misread to Avoid: Filtering out near-zero compound reviews before checking ambivalence. High-ambivalence reviews often contain the most actionable product tradeoff feedback.

📐 Technical Definition: When both positive and negative proportions equal zero, ambivalence is defined as 0.00. That separates genuinely neutral sentences from conflicted ones.

🎯 The Manager's Question: How much of the effect runs through the first mediator and then the second, in that order?

📊 Worked Business Example: In the Northwind Outdoors email case, email lifts awareness (0.51), awareness lifts consideration (0.46) and consideration lifts purchase (0.49). The chained route is 0.51 × 0.46 × 0.49 = 0.115, the largest point estimate of the three routes.

⚖️ How to Judge the Number: Zero means nothing travels down the chain, and its interval against 0 is the test. Read its size next to the two single-mediator routes and the total indirect effect: 0.09, 0.07 and 0.27 in the email case. Calling it the biggest route needs its contrast with each other route to clear zero; against the awareness route it runs from -0.13 to 0.06. A chain with the largest share, 43% here, supports a funnel plan that builds awareness first, if the order came from theory or timing.

⚠️ Common Misread to Avoid: A chained route that clears zero is read as evidence that the first mediator comes before the second. It is not. Reversing the two mediators gives a different model that fits the same data equally well, so the order has to come from theory or from when each was measured.

📐 Technical Definition: Assumes the order of the mediators is right and nothing unmeasured links them. The tool fits PROCESS model 6 with two mediators, which has three routes: \(a_1 b_1\), \(a_2 b_2\) and the chain. They add to the total indirect effect, and each route and each contrast between routes gets its own Monte Carlo or bootstrap interval.

🎯 The Manager's Question: How many times was our ad delivered under the counting rule?

📊 Worked Business Example: At a $5 CPM, the $10,000 budget buys $10,000 ÷ $5 × 1,000 = 2,000,000 served impressions. At an average frequency of 2, that is about 1,000,000 different people.

⚖️ How to Judge the Number: It is a count, so it starts at 0 and has no good or bad level on its own. Compare it against reach: 2,000,000 impressions over 1,000,000 people means each saw it twice on average. Read it next to viewability, since served is not the same as seen.

⚠️ Common Misread to Avoid: Reading 2,000,000 impressions as 2,000,000 people. One person can account for many impressions; at an average frequency of 5 the same total reaches only 400,000.

📐 Technical Definition: The tool treats this as an expected count from the plan's assumptions, not a delivery report. It assumes every served impression is billable and measurable.

🎯 The Manager's Question: Do the model's misses follow a bell shape, or are they lopsided or heavy at the ends?

📊 Worked Business Example: For the 60 ad campaigns W = 0.951 and p = 0.017, so the misses are less bell-shaped than chance allows. The Q-Q plot shows why: the three smallest budgets fell far below the line.

⚖️ How to Judge the Number: W runs up to 1, and a p-value below {alpha} says the misses are less bell-shaped than chance allows. Read it with the Q-Q plot and the sample size: with 30 or more rows, mild departures matter little for the coefficients. The tool treats the failure as severe only below 30 rows, or with skewness beyond 2 or excess kurtosis above 3.

⚠️ Common Misread to Avoid: Running it on {outcome} instead of the residuals. The assumption is about the misses, and even when {outcome} itself is skewed the misses can be bell-shaped. With large samples the test also flags departures too small to matter.

📐 Technical Definition: The weights use Royston's approximation, which matches scipy.stats.shapiro. The tool skips the test with fewer than 3 or more than 5,000 residuals.

🎯 The Manager's Question: How much of this portfolio's audience should each item get credit for?

📊 Worked Business Example: Frost & Fold's best three flavors reach 85.3% of shoppers. Chocolate Fudge Brownie earns 35.8 points of that, Raspberry Ripple 25.1 and Strawberries & Cream 24.4.

⚖️ How to Judge the Number: The credits always add up to the portfolio's reach, and 0 means an item reaches no one in the set. A shopper who likes two of the three counts half to each. Read it next to the removal loss: Brownie earns 35.8 points of credit, but dropping it costs only 24.4, because some of its fans like another pick too.

⚠️ Common Misread to Avoid: Picking the highest-credit items as the best portfolio. Credit is shared out within this portfolio only, and a different set changes every item's credit. The search that found the set, not the credits, says which items to launch.

📐 Technical Definition: The sum runs over respondents who accept item j. In a coverage game this equal split is the exact Shapley value, so no orderings need to be sampled. The tool prints it as a percentage of all respondents.