Chapter 10 · Relationships

Regression & Correlation

When crude goes up a dollar, how much does gasoline at the pump go up? Regression answers with a line, a slope and an honest interval — and residual analysis tells you when the line is lying. Real data: 435 months of U.S. retail gasoline and Brent crude.

~45 min Data: EIA retail regular gasoline (weekly → monthly) · EIA Brent · World Bank Pink Sheet

In 60 seconds

The least-squares line ŷ = b₀ + b₁x minimises the sum of squared residuals.
Slope b₁: change in y per one-unit change in x. R²: share of the variation in y explained by x. r: strength and direction of the linear relationship.
For inference, check four assumptions: linearity, constant spread, normal errors and independent errors — time series usually fail the last.
A prediction interval (one future value) is much wider than a confidence interval for the mean response.

Lines, slopes and intercepts

A straight line y = b₀ + b₁x has y-intercept b₀ (value of y at x = 0) and slope b₁ (rise per unit run: positive slopes go up, negative down, zero is flat). In regression, x is the predictor (explanatory variable) and y the response.

From crude to the pump

U.S. regular gasoline vs Brent crude, monthly averages

1990-08 to 2026-10 · each dot is a month · orange line = least-squares fit · band = 95 % prediction interval

Least-squares slope

b_1 = \frac{S_{xy}}{S_{xx}} = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}

Intercept

b_0 = \bar{y} - b_1 \bar{x}

The fitted line always passes through (\bar{x}, \bar{y}).

Intercept b₀

$0.707

per gallon at zero crude cost

Slope b₁

0.0287

$/gal per $/bbl

R²

93.3%

of pump-price variation

Correlation r

0.966

strong, positive, linear

Reading the slope: each extra dollar per barrel of Brent goes with about 2.9 cents per gallon at the pump. A barrel holds 42 gallons, so pure pass-through of crude cost would be 1/42 = 2.38 ¢. The fitted slope implies only 34.8 "gallons per barrel" — more than full pass-through. Why? Over 36 years taxes, refining margins and general inflation rose at the same time as crude prices, and the one-variable line credits all of it to crude. Chapter 12 adds time as a second predictor and the implied figure moves to ≈ 43.5 — almost exactly the physical 42.

The intercept (~71 ¢ at $0 crude) is an extrapolation — Brent never traded near zero in this sample. Only interpret predictions inside the range of the x data.

How good is the fit? Sums of squares and R²

The coefficient of determination R² = 0.933: the Brent price accounts for 93 % of the variation in monthly pump prices. In simple linear regression R² equals the square of the correlation r.

Correlation

Linear correlation coefficient (Pearson)

r = \frac{S_{xy}}{\sqrt{S_{xx}\, S_{yy}}}

-1 \le r \le 1. Sign = direction, size = strength of the linear pattern. Unit-free and symmetric in x and y.

Brent vs WTI monthly returns

r = 0.95 · two benchmarks, one market

Brent vs U.S. natural gas monthly returns, 2000+

r = 0.12 · weakly related

U.S. gas (Henry Hub) has largely decoupled from oil since the shale era. Still, with n = 299 months the small correlation is statistically significant (p = 0.037) — significant, yet explaining only r² ≈ 1.5 % of the variation. Two reminders: correlation measures only linear association, and it never proves causation — a lurking variable (global demand, the dollar) can drive both.

Correlation t-test (H₀: ρ = 0)

t = \frac{r \sqrt{n - 2}}{\sqrt{1 - r^2}}, \qquad df = n - 2

Regression inference

The sample line estimates a population regression line μ_y|x = β₀ + β₁x. Inference needs the regression model assumptions:

Linearity: the conditional means of y lie on a straight line.
Equal SD: the conditional SD σ of y is the same for every x.
Normality: for each x, y is normally distributed around the line.
Independence: the errors are independent of each other.

Standard error of the estimate

s_e = \sqrt{\frac{SSE}{n - 2}}

Estimates σ: here $0.252 per gallon — the typical miss of the line.

Slope t-test & interval

t = \frac{b_1}{s_e / \sqrt{S_{xx}}}, \qquad b_1 \pm t_{\alpha/2} \cdot \frac{s_e}{\sqrt{S_{xx}}}

df = n − 2. Here t = 77.7; 95 % CI for β₁: 0.0280 to 0.0295.

Confidence interval vs prediction interval

CI for the conditional mean at x₀

\hat{y} \pm t_{\alpha/2}\, s_e \sqrt{\frac{1}{n} + \frac{(x_0 - \bar{x})^2}{S_{xx}}}

"Average pump price over all months with Brent at $80."

Prediction interval at x₀

\hat{y} \pm t_{\alpha/2}\, s_e \sqrt{1 + \frac{1}{n} + \frac{(x_0 - \bar{x})^2}{S_{xx}}}

"Pump price in one particular month with Brent at $80." The extra 1 is the month's own scatter.

At Brent = $80/bbl: predicted $3.01/gal; 95 % CI for the mean 2.98 – 3.04, but 95 % prediction interval 2.51 – 3.50. Forecasting a single month is far less certain than estimating an average.

Residual analysis: is the model trustworthy?

Residuals vs fitted values

Should be a structureless band around 0

Residuals over time

Should look like random noise

The right panel fails loudly: residuals drift in long waves — close to zero in the 1990s, swinging negative around 2009–2016, and strongly positive in the 2020s — instead of bouncing randomly. The Durbin–Watson statistic is 0.22 (about 2 means independent errors; near 0 means strong positive autocorrelation). Consequence: the slope estimate is still reasonable, but its standard error, t-statistic and intervals are far too optimistic. The fix — model the trend and the time dependence — is chapter 12.

Outliers and influential observations

An outlier sits far from the line (large residual); an influential observation changes the line noticeably when removed — typically a point with an extreme x. Spikes like summer 2008 or 2022 have large x and residuals: refit without them and compare before trusting any slope.

Do it in code

select count(*)                         as n,
       regr_intercept(gasoline, brent)  as b0,      -- 0.7066
       regr_slope(gasoline, brent)      as b1,      -- 0.02874
       regr_r2(gasoline, brent)         as r2,      -- 0.9331
       corr(gasoline, brent)            as r,       -- 0.9660
       regr_sxx(gasoline, brent)        as sxx
from gasoline_crude_monthly;

import statsmodels.formula.api as smf
m = smf.ols("gasoline ~ brent", data=gm).fit()
m.params, m.rsquared, m.conf_int()                      # b0, b1, R², CI for β1
m.get_prediction(pd.DataFrame({"brent": [80]})).summary_frame(alpha=0.05)   # CI and PI at $80
sm.stats.stattools.durbin_watson(m.resid)               # 0.22 -> autocorrelated errors
stats.pearsonr(np.log(wb.brent).diff().dropna(), np.log(wb.gas_us).diff().dropna())

Key takeaways

Least squares gives b₁ = S_xy/S_xx; interpret the slope in real units and do not extrapolate.
SST = SSR + SSE; R² is the explained share; in simple regression R² = r².
Prediction intervals for single values are much wider than confidence intervals for the mean.
Plot residuals against fitted values and time. Autocorrelated residuals invalidate the usual inference.
A one-variable slope can absorb the effect of omitted variables that move with x.

Self-check

1 · Predict

Using ŷ = 0.7066 + 0.02874x, predict the pump price for Brent at $100. Is that interpolation?

Answer

0.7066 + 2.874 = $3.58/gal. Brent above $100 occurred in the sample (2008, 2011–14, 2022), so it is inside the data range — interpolation.

2 · Interpret R²

What does R² = 0.933 not tell you?

Answer

It says nothing about causation, about whether a straight line is the right form, about the independence of errors, or about the accuracy of a single forecast (that needs s_e and the prediction interval).

3 · Significant but weak

Brent–gas correlation r = 0.12 is significant at 5 %. Should a gas trader hedge gas with Brent futures?

Answer

Hardly: r² ≈ 0.015 — Brent explains about 1.5 % of gas return variation, so the hedge would remove almost no risk. Significance is not usefulness.