Chapter 08 · Inference

Comparing Two Groups

Most real questions are comparisons: this season vs that one, before vs after, crude A vs crude B. The test engine from chapter 07 stays the same; what changes is how the two samples relate to each other — and that choice matters more than the formula.

~40 min Data: EIA PADD 3 refinery utilisation · Brent & WTI · NCS wildcats

In 60 seconds

Independent samples (two separate groups) → Welch t-test (default) or pooled t if the SDs are equal; Mann–Whitney when normality fails.
Paired samples (two measurements on the same unit or date) → analyse the differences with a paired t-test or paired Wilcoxon. Pairing removes shared noise and is far more powerful.
Compare spreads with the F-test (fragile) or Levene's test (robust); compare rates with the two-proportion z-test.

Independent or paired?

Independent samples: pooled and Welch t

Gulf Coast (PADD 3) refineries run harder in summer driving season than in winter maintenance season — but by how much, and is it beyond noise? Weekly utilisation since 2010, June–August vs December–February:

PADD 3 (Gulf Coast) weekly refinery utilisation by season, 2010 – today

% of operable capacity · dots = mean

Standard error of x̄₁ − x̄₂

SE(\bar{x}_1 - \bar{x}_2) = \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}

The difference of two independent means has variance \sigma_1^2/n_1 + \sigma_2^2/n_2.

Welch (non-pooled) t

t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\dfrac{s_1^2}{n_1} + \dfrac{s_2^2}{n_2}}}

df from the Welch–Satterthwaite formula. The safe default.

Pooled t (σ₁ = σ₂ assumed)

\begin{gathered} s_p = \sqrt{\frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}} \\[6pt] t = \frac{\bar{x}_1 - \bar{x}_2}{s_p \sqrt{1/n_1 + 1/n_2}} \end{gathered}

df = n_1 + n_2 - 2. Only if the SDs are similar (ratio under ~2).

Summer mean 93.2 % vs winter 89.2 %; Welch t = 7.57 (pooled t = 7.68), p < 0.0001. The winter SD (6.4) is larger than summer's (4.3): winter includes freezes (February 2021) and turnarounds — a reason to prefer Welch.

An assumption to question

These tests assume independent observations. Consecutive weeks of the same refinery system are strongly correlated, so the effective sample size is much smaller than 429 and the true p-value larger. The seasonal difference is still real (it shows up every year), but chapter 12 shows how to handle the dependence properly.

Mann–Whitney: comparing without normality

Are Barents Sea wildcats drilled faster or slower than North Sea ones? Drilling times are skewed with long right tails, so use the Mann–Whitney (Wilcoxon rank-sum) test: pool both samples, rank them, and check whether one group's ranks run systematically high. It assumes the two distributions have the same shape (one is just shifted).

Drilling days of wildcats spudded since 2000, by area

Days from spud to completion · dots = mean

Medians: North Sea 40 days, Barents Sea 38 days. Mann–Whitney p = 0.55: no evidence of a difference in typical drilling time — despite harsher Arctic conditions.

Paired samples: the Brent–WTI spread

Brent (North Sea) and WTI (Cushing, Oklahoma) are priced on the same days and share almost every global shock. Comparing them as two independent samples would drown the difference in that shared noise. Instead take the paired difference d = Brent − WTI each month and run a one-sample test on d.

Monthly average spread, Brent minus WTI

USD per barrel · positive = Brent premium · shading = before 2011

Paired t-test

t = \frac{\bar{d}}{s_d / \sqrt{n}}, \qquad df = n - 1

Needs the differences (not the raw values) to be roughly normal or n large. Non-normal but symmetric differences → paired Wilcoxon signed-rank test.

Mean spread to 2010

−1.37

USD/bbl · n = 284 months · t = −17.7

Mean spread since 2011

+6.56

USD/bbl · n = 190 months · t = 15.7

Both eras give overwhelming evidence (p < 0.0001) — in opposite directions. Until 2010 WTI, a lighter and sweeter crude, traded at a premium. After the U.S. shale boom filled Cushing faster than pipelines could drain it, Brent took a persistent premium. A test on the pooled 1987–today data would average two different regimes into a meaningless number: always look at the series before testing it.

Comparing spreads: the F-test

Was the oil market more volatile in 2015–2019 than in 2010–2014? The ratio of two sample variances follows an F-distribution (two df parameters, right-skewed, always ≥ 0) when both populations are normal.

Two-standard-deviations F-test

F = \frac{s_1^2}{s_2^2}, \qquad df = (n_1 - 1,\ n_2 - 1)

Reciprocal property: the left-tail critical value of F(a, b) is 1 / (right-tail value of F(b, a)).

F(59, 59) null distribution and the observed ratio

Monthly Brent return volatility: s = 9.34 % (2015–19) vs 6.18 % (2010–14)

F = 2.28, two-tailed p = 0.0018. The F-test is notoriously sensitive to non-normality, and these returns are fat-tailed — so check with Levene's test, which compares average absolute deviations instead: p = 0.015. Same conclusion, more trustworthy evidence: the 2014–2016 price collapse ushered in a more volatile period.

Comparing two proportions

Did Norwegian wildcats succeed more often in the 2010s than in the 1990s?

Two-proportions z-test

z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}_p (1 - \hat{p}_p) \left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}

Pooled proportion \hat{p}_p = (x_1 + x_2)/(n_1 + n_2), because H₀ says the two rates are equal.

CI for p₁ − p₂

(\hat{p}_1 - \hat{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\hat{p}_1(1 - \hat{p}_1)}{n_1} + \frac{\hat{p}_2(1 - \hat{p}_2)}{n_2}}

Unpooled SE for the interval. Plus-four version: add 1 success and 1 failure to each sample.

1990s: 44.5% of 247 wells; 2010s: 47.9% of 334. z = 0.81, p = 0.42. 95 % CI for the change: −4.8% to 11.6%. The data are consistent with no change — and with a gain of up to 12 points. A decade of drilling cannot resolve a few-point improvement.

Which two-sample procedure?

Design	Data look…	Use
Independent	normal-ish or large n	Welch t (pooled t only if SDs are equal)
Independent	skewed / outliers, same shapes	Mann–Whitney
Paired	differences normal-ish or large n	paired t
Paired	differences symmetric, outliers	paired Wilcoxon signed-rank
Two SDs	both normal	F-test (else Levene / Brown–Forsythe)
Two rates	at least 5 successes and failures per group	two-proportions z (plus-four for the CI if small)

Do it in code

stats.ttest_ind(summer, winter, equal_var=False)   # Welch: t = 7.57
stats.ttest_ind(summer, winter, equal_var=True)    # pooled: t = 7.68
stats.mannwhitneyu(days_north, days_barents)        # p = 0.55
stats.ttest_rel(brent_post2011, wti_post2011)       # paired: t = 15.7
stats.wilcoxon(brent_post2011 - wti_post2011)       # paired Wilcoxon
F = v2.var(ddof=1) / v1.var(ddof=1)                 # 2.28
2 * min(stats.f.sf(F, 59, 59), stats.f.cdf(F, 59, 59))   # p = 0.0018
stats.levene(v1, v2)                                # p = 0.015
from statsmodels.stats.proportion import proportions_ztest
proportions_ztest([x2010s, x1990s], [334, 247])     # z = 0.81, p = 0.42

select case when extract(year from month) <= 2010 then '<= 2010' else '>= 2011' end as era,
       count(*) as n,
       avg(brent - wti) as mean_spread,
       stddev_samp(brent - wti) as sd_spread,
       avg(brent - wti) / (stddev_samp(brent - wti) / sqrt(count(*))) as paired_t
from v_brent_monthly
group by 1 order by 1;

Key takeaways

First decide: independent or paired. Pairing removes shared noise.
Welch's t is the default for independent means; rank tests when distributions are skewed.
The F-test for variances breaks under fat tails; confirm with Levene.
Always plot the series: regime changes (the 2011 spread flip) can make a pooled test meaningless.

Self-check

1 · Design choice

You compare the yield of 30 crude cargoes tested in two labs (each cargo tested in both). Independent or paired?

Answer

Paired — each cargo is measured twice. Analyse the 30 lab-to-lab differences with a paired t-test.

2 · Read the interval

The 95 % CI for p₂₀₁₀ₛ − p₁₉₉₀ₛ is −4.8 % to 11.6 %. What does it say about the test of equal rates?

Answer

The interval contains 0, so the two-tailed test at α = 0.05 does not reject equality — consistent with p = 0.42.

3 · Pooled or Welch?

Group SDs are 6.4 and 4.3 with sizes 206 and 223. Which t-test?

Answer

Welch. The SD ratio (1.5) suggests unequal variances; Welch loses almost nothing when they are equal and protects you when they are not.