Linear Regression

Study guides: Exam MAS-I, Exam MAS-II, Exam DISC-DA

Linear Regression models a continuous response as a linear function of predictors plus Normal error, with coefficients chosen to minimize the Residual Sum of Squares (ordinary least squares, OLS). It is the base case of the Extended Linear Model family: a Generalized Linear Model with a Normal response and the identity Link Function.

$Y_i = \beta_0 + \beta_1 x_{i1} + \cdots + \beta_p x_{ip} + \varepsilon_i, \qquad \varepsilon_i \stackrel{\text{iid}}{\sim} N(0, \sigma^2)$

$\hat{\boldsymbol\beta} = (\mathbf{X}^{\top}\mathbf{X})^{-1}\mathbf{X}^{\top}\mathbf{y}$

The four assumptions — linearity, independence, constant variance, normality of errors — are exactly what the diagnostic plots check: a Residual Plot for the first and third, a QQ Plot for the fourth
$\hat\beta_j$ is the change in $E[Y]$ per unit of $x_j$ holding the other predictors fixed; with Multicollinearity present that clause is what makes coefficients unstable
Fit is summarized by R-Squared $= 1 - \text{RSS}/\text{SS}_{\text{Tot}}$ , with $\hat\sigma^2 = \text{RSS}/(n - p - 1)$ estimating the error variance
Leaving out $\beta_0$ , for instance when its test shows it is not significantly different from zero, gives regression through the origin: the fitted response is then zero whenever every predictor is zero
OLS is the MLE under Normal errors, and by Gauss–Markov it is the minimum-variance linear unbiased estimator whenever the errors have mean zero, are uncorrelated and have constant variance ( $E[\boldsymbol\varepsilon] = \mathbf{0}$ , $\text{Var}(\boldsymbol\varepsilon) = \sigma^2\mathbf{I}$ ) — normality is needed only for exact tests and intervals in small samples
Insurance data usually violates the assumptions: severity is right-skewed and its variance grows with the mean, frequency is a non-negative count. That is precisely the motivation for the GLM, not a reason to keep transforming the response
The overall test that all slopes are zero is the $F$ test in the ANOVA table; individual coefficients are tested in Parameter Estimate Tables

Fitting a Simple Regression For a simple regression with $n = 20$ , $S_{xx} = 250$ , $S_{xy} = 400$ , $\bar{x} = 12$ , $\bar{y} = 45$ . Find the fitted line and the predicted value at $x = 15$ .

Answer $\hat\beta_1 = \frac{S_{xy}}{S_{xx}} = \frac{400}{250} = 1.6$ $\hat\beta_0 = \bar{y} - \hat\beta_1\bar{x} = 45 - 1.6(12) = 25.8$ $\hat{y} = 25.8 + 1.6x \ \Longrightarrow\ \hat{y}(15) = 25.8 + 24 = 49.8$

When Linear Regression Is the Wrong Tool An actuary regresses claim counts (mostly $0$ , $1$ , or $2$ ) on exposure and territory with OLS. The Residual Plot fans out and some fitted values are negative. What has gone wrong and what should replace it?

Answer Two assumptions fail at once. A count response has variance that grows with the mean, breaking constant variance, and the identity link lets the linear predictor run below zero, producing impossible negative expected counts. The fix is a Poisson GLM with a log link (Poisson Regression), with log exposure as an offset — the log link keeps $\hat\mu > 0$ and the Poisson variance function $V(\mu) = \mu$ matches the observed spread.

How it appears on the exam

DISC-DA: Section 8.3 covers analyzing data with linear functions. Questions give a fitted line or a short coefficient table and ask for a prediction, the meaning of a slope holding the other predictors fixed, or whether a linear model suits the response, as in the claim-count example above.

MAS-I and MAS-II: Linear regression is the base case of the linear model family: least squares estimates, $R^2$ , the $F$ test and the diagnostic plots that check its assumptions.

Exam traps

Slope without "holding the others fixed": In a multiple regression, $\hat\beta_j$ is the change in the mean response for a unit change in $x_j$ with the other predictors unchanged, not the one-variable relationship.

Counts and skewed losses: A Normal linear model can predict negative claim counts and ignores variance that grows with the mean; a GLM is the usual fix.