Chapter 12 · Hero Level

Hero Level: Real-World Methods

The classical procedures assume normal populations, independent observations, one test at a time and one predictor at a time. Real energy data breaks all four. This chapter gives you the tools that experienced analysts reach for when that happens — each demonstrated on a problem the earlier chapters left open.

~60 min Data: all datasets of the track

In 60 seconds

Bootstrap: resample your own data to get standard errors and intervals for any statistic, without normality.
Permutation tests: shuffle labels to build the null distribution directly.
Multiple testing: many tests produce false discoveries; correct with Bonferroni or Benjamini–Hochberg.
Autocorrelation: time-series observations are not independent; the effective sample size can be tiny.
Multiple regression controls for confounders; logistic regression models yes/no outcomes; Bayesian updating combines prior knowledge with new data.

1 · The bootstrap

Chapter 05 needed σ and a normal sampling distribution. The bootstrap needs neither: treat the sample as a stand-in for the population, draw many resamples of the same size with replacement, recompute the statistic each time, and read the spread of those values. The middle 95 % of the bootstrap distribution is a percentile bootstrap interval.

Server result with 10,000 resamples: median field size 18.7, 95 % bootstrap CI 14.6 – 25.2 million Sm³ o.e. — asymmetric around the estimate, as it should be for skewed data. For volatility, the bootstrap interval comes out noticeably wider than the χ² interval of chapter 06: fat tails make the textbook interval over-confident.

2 · Permutation tests

To test whether the 1990s and 2010s discovery rates differ, a permutation test asks directly: if decade did not matter, how often would randomly relabelled wells show a difference this large? Pool the wells, shuffle the decade labels, recompute the difference, repeat thousands of times.

Server: observed difference 3.4%, permutation p = 0.45 (z-test in chapter 08: 0.42). Permutation tests make no distributional assumption, work for any statistic (medians, ratios, correlation), and are exact for small samples.

3 · Multiple testing: the "seasonal trading" trap

"Oil rallies in some months" — test each calendar month separately: is the mean monthly Brent return different from zero?

p-values of 12 one-sample t-tests (one per calendar month)

Monthly Brent returns since 1987 · reference lines at α = 0.05 and Bonferroni α/12 = 0.0042

One month (November) clears p < 0.05 — exactly what chance alone predicts when you run 12 tests (expected false positives: 12 × 0.05 = 0.6). After correction it vanishes: Bonferroni-adjusted p = 0.34, Benjamini–Hochberg (false-discovery-rate) adjusted p = 0.34.

Bonferroni

\text{reject } H_{0,i} \text{ if } p_i \le \frac{\alpha}{m}

Controls the chance of any false positive. Simple, conservative.

Benjamini–Hochberg

\begin{gathered} \text{sort } p_{(1)} \le \dots \le p_{(m)} \\ i^* = \max\{ i : p_{(i)} \le \tfrac{i}{m}\,\alpha \} \\ \text{reject } H_{(1)}, \dots, H_{(i^*)} \end{gathered}

Controls the expected share of false discoveries. More power when testing many things.

4 · Autocorrelation and effective sample size

Weekly refinery utilisation this week is almost the same as last week. Observations that "remember" each other carry less information than independent ones.

Autocorrelation of U.S. weekly refinery utilisation

Correlation between the series and itself shifted by k weeks (lag 1–52)

Lag-1 autocorrelation ρ₁ = 0.93. For a first-order process the effective sample size is roughly n·(1 − ρ₁)/(1 + ρ₁): the 1,862 weeks carry the information of about 70 independent observations. A naive standard error is therefore too small by a factor √(1862/70) ≈ 5. That is why chapter 08's seasonal t of 7.6 and chapter 10's slope t of 78 overstate the certainty.

What to do instead

Aggregate to independent-ish units (one value per year, per field, per well).
Use autocorrelation-robust (Newey–West / HAC) standard errors in regression.
Model the dependence explicitly (ARIMA and other time-series models).
Work with changes or returns, not levels: prices behave like random walks, and regressing one random walk on another produces "spurious" high R² values.

5 · Multiple regression and omitted-variable bias

Chapter 10's slope implied 34.8 "gallons per barrel" because taxes, margins and inflation rose over time together with crude prices. Add time (years since 2000) as a second predictor:

Multiple linear regression

\hat{y} = b_0 + b_1 \cdot \text{Brent} + b_2 \cdot t

Each slope is the effect of its variable holding the others constant.

Brent slope b₁

0.0230

$/gal per $/bbl

Implied gallons / barrel

43.5

physical barrel: 42

Time slope b₂

+0.025

$/gal per year, at fixed crude price

R²

97.1%

vs 93.3 % with Brent alone

Controlling for time, crude pass-through matches the physics: one barrel's price spread over ≈ 43 gallons. The other 2.5 ¢ per year is the slow upward drift of everything that is not crude — taxes, refining and distribution margins, inflation. This is omitted-variable bias in action: leave out a variable that moves with x and affects y, and x's slope absorbs its effect.

6 · Decline curves: regression on a log scale

Oil-field output typically falls exponentially after the plateau: q(t) = q₀·e^−Dt. Taking logs turns it into a straight line, ln q = ln q₀ − D·t, so the decline rate is just a regression slope.

7 · Logistic regression: modelling yes / no

Can water depth and sea area predict whether a wildcat finds hydrocarbons? The outcome is binary, so model the log-odds: ln(p / (1 − p)) = b₀ + b₁x₁ + …. Each coefficient becomes an odds ratio e^b.

Term	Coefficient	Odds ratio	p

Fitted on 744 wildcats since 2000: every odds ratio is close to 1 and no p-value is small. Neither water depth nor area predicts success — consistent with chapters 03 and 09. A well-specified model that finds nothing is a valuable, honest result: success depends on prospect-level geology these variables do not capture.

8 · Bayesian updating

Only 55 Barents Sea wildcats were drilled 2015–2024, with 29 discoveries. A Bayesian analysis starts from a prior belief about the success rate p, expressed as a Beta(a, b) distribution, and updates it with the data: the posterior is Beta(a + successes, b + failures).

Beta–binomial update

\begin{gathered} \text{prior } \text{Beta}(a, b) \\ \big\downarrow \ x \text{ successes in } n \\ \text{posterior } \text{Beta}(a + x,\ b + n - x) \end{gathered}

Prior mean a/(a + b); the prior "weighs" like a + b earlier wells.

With a modest prior (≈ 37 virtual wells, the server default) the posterior mean is 50.2% with a 95 % credible interval 40.0% – 60.3%, pulled slightly from the raw 52.7% towards the prior. Unlike a confidence interval, a credible interval does mean "given prior and data, p lies here with 95 % probability" — at the price of having to state the prior openly.

Where to go next

Topic	Energy use case
Time-series models (ARIMA, GARCH)	Price and volatility forecasting; demand forecasting
Generalised linear models (Poisson, negative binomial)	Incident and failure counts with overdispersion (chapter 04)
Survival analysis	Time to equipment failure, well abandonment
Design of experiments	Catalyst, additive and process-setting trials
Hierarchical (multilevel) Bayesian models	Success rates for many plays sharing information
Monte Carlo simulation	Reserves (P90/P50/P10) and project economics

Do it in code

rng = np.random.default_rng(0)
boot = [np.median(rng.choice(f, f.size)) for _ in range(10_000)]
np.percentile(boot, [2.5, 97.5])                                   # bootstrap CI for the median
stats.permutation_test((a, b), lambda x, y: y.mean() - x.mean(), n_resamples=10_000)
from statsmodels.stats.multitest import multipletests
multipletests(pvals, method="fdr_bh")                              # Benjamini–Hochberg
from statsmodels.tsa.stattools import acf; acf(util, nlags=52)
smf.ols("gasoline ~ brent + t", gm).fit(cov_type="HAC", cov_kwds={"maxlags": 12})   # Newey–West SEs
smf.logit("discovery ~ wd100 + C(main_area)", wells).fit().summary()
stats.beta(a + x, b + n - x).ppf([0.025, 0.975])                   # Bayesian credible interval

Key takeaways

Bootstrap and permutation methods replace distributional assumptions with computation.
Many tests → correct for multiplicity, or false discoveries are guaranteed.
Time-series data are dependent; compute the effective sample size before trusting p-values.
Control for confounders with multiple regression; model binary outcomes with logistic regression.
Bayesian updating makes prior knowledge explicit and gives directly interpretable intervals.

Self-check

1 · Bootstrap or t?

You need a 95 % interval for the 90th percentile of field sizes. Which method?

Answer

The bootstrap: resample, compute the 90th percentile each time, take the 2.5th–97.5th percentiles of those values. There is no simple t-type formula for a percentile.

2 · Twenty tests

You test 20 refinery KPIs for a change after a software upgrade; two have p < 0.05. Conclusion?

Answer

Expected false positives under no change: 20 × 0.05 = 1. Two "hits" are weak evidence. Apply Bonferroni (α/20 = 0.0025) or Benjamini–Hochberg and confirm with fresh data.

3 · Prior strength

What happens to the Bayesian posterior as the prior weight grows to thousands of virtual wells?

Answer

The posterior sticks to the prior mean and the Barents data barely move it. Strong priors need strong justification — here, evidence that Barents geology behaves like the North Sea's.