Chapter 11 · Relationships

Analysis of Variance (ANOVA)

To compare the means of three or more groups at once, compare two kinds of variation: how far the group means sit from each other versus how much the data scatter inside each group. If the first is large relative to the second, the groups differ.

Why not just run several t-tests?

With three groups you would need 3 pairwise t-tests; with 5 groups, 10. Each at α = 0.05 means the chance of at least one false alarm grows quickly — about 1 − 0.95¹⁰ ≈ 40 % for ten independent tests. ANOVA asks one question with one α.

The idea in one picture

Water depth across Norway's three seas

Does typical water depth at wildcat locations differ between the North Sea, Norwegian Sea and Barents Sea (wells spudded since 2000)?

Sums of squares and the ANOVA table

F = 147.0, p < 0.0001: mean water depth differs between the areas. The share of total variation explained by area, η² = SSTR/SST = 28.4%, is ANOVA's equivalent of R².

Checking the assumptions

1. Independent simple random samples from each population.
2. Normal populations — check with normal probability plots of the residuals (value minus its group mean).
3. Equal standard deviations — rule of 2: OK if the largest sample SD is at most twice the smallest.

Which groups differ? Tukey multiple comparisons

A significant F does not say which means differ. Tukey's method builds confidence intervals for every pair of means while holding the family confidence level (all intervals correct simultaneously) at 95 %, using the studentised range (q) distribution. Each individual interval is therefore slightly wider than a stand-alone t-interval.

All three pairs differ: the North Sea is shallowest, the Barents Sea in between, the Norwegian Sea deepest on average.

Kruskal–Wallis: the rank-based alternative

The Kruskal–Wallis test ranks all observations together and checks whether the average rank differs between groups. It needs only that the distributions have the same shape — no normality. Here H = 258.2 (compared with χ² on 2 df), p < 0.0001: the conclusion survives without the normality assumption.

A second example: refinery regions

Mean weekly utilisation in 2023 by PADD region: . One-way ANOVA: F = 12.1, p < 0.0001. But the five regions are measured in the same 52 weeks — a hurricane week hits several regions at once — so the samples are not independent. A design that pairs regions by week (a "randomized block" analysis, the multi-group cousin of the paired t-test) is the correct tool; one-way ANOVA here is only descriptive.

Do it in code

w = pd.read_sql("select main_area, water_depth_m from v_wildcats where entry_year >= 2000 and water_depth_m is not null", con)
groups = [g.water_depth_m for _, g in w.groupby("main_area")]
stats.f_oneway(*groups)                     # F = 147.0, p ≈ 0
stats.kruskal(*groups)                      # H = 258.2
model = smf.ols("water_depth_m ~ C(main_area)", data=w).fit()
sm.stats.anova_lm(model)                    # the ANOVA table
from statsmodels.stats.multicomp import pairwise_tukeyhsd
pairwise_tukeyhsd(w.water_depth_m, w.main_area)

with g as (select main_area, count(*) n, avg(water_depth_m) m, var_samp(water_depth_m) v
           from v_wildcats where entry_year >= 2000 and water_depth_m is not null
           group by main_area),
     o as (select sum(n * m) / sum(n) as grand from g)
select sum(n * (m - grand)^2)                                   as sstr,
       sum((n - 1) * v)                                          as sse,
       (sum(n * (m - grand)^2) / (count(*) - 1))
         / (sum((n - 1) * v) / (sum(n) - count(*)))              as f
from g, o;

Self-check