Deviance

Study guides: Exam MAS-I, Exam PCPA

Deviance is a goodness-of-fit measure for Generalized Linear Models based on the log-likelihood. It compares the fitted model to the saturated model (a model with one parameter per observation, achieving perfect fit).

$D^* = 2\left[\ell_{\text{sat}} - \ell(\hat{\boldsymbol{\beta}})\right] = 2\left[\ell(\text{saturated model}) - \ell(\text{fitted model})\right]$

$D = \phi\, D^*, \qquad D^* = \frac{D}{\phi}$

$D^*$ is the scaled deviance and $D$ , the scaled deviance multiplied by the Dispersion Parameter $\phi$ , the unscaled deviance. $D$ does not depend on $\phi$ , which makes it the one to compare models with different dispersion estimates. For Poisson and binomial models $\phi = 1$ and the two coincide
The names are not standard: these follow McCullagh & Nelder and CAS Monograph 5, while Dobson & Barnett call $2[\ell_{\text{sat}} - \ell]$ the "deviance" and its product with $\hat\phi$ the "scaled deviance" — the reverse. ISLR calls $-2\ell(\hat{\boldsymbol{\beta}})$ itself the deviance. Read which one a question means
Deviance is always non-negative; the saturated model has deviance $0$ (a perfect fit)
For a Normal linear model the unscaled deviance is the residual sum of squares, $D = \text{RSS} = \sum(y_i - \hat{y}_i)^2$ , so $D^* = \text{RSS}/\sigma^2$
For Poisson regression ( $\phi = 1$ ): $D = 2\sum_i \left[y_i\ln\!\frac{y_i}{\hat{\mu}_i} - (y_i - \hat{\mu}_i)\right]$ ; in large samples a model that fits has $D \approx \chi^2_{n-p}$
Analysis of Deviance: for nested models, $\dfrac{D_{\text{reduced}} - D_{\text{full}}}{\phi} \approx \chi^2_{\Delta p}$ when $\phi$ is known (large samples). When $\phi$ is estimated, use $F = \dfrac{D_{\text{reduced}} - D_{\text{full}}}{\Delta p\,\hat\phi_{\text{full}}}$ on $(\Delta p,\ n - p_{\text{full}})$ degrees of freedom

Analysis of Deviance to Test a Predictor A Poisson GLM with intercept only has deviance $D_0 = 45.2$ on 49 df. Adding "territory" (3 levels, 2 extra parameters) gives $D_1 = 30.8$ on 47 df. Test whether territory significantly improves fit at $\alpha = 0.05$ .

Answer $\Delta D = 45.2 - 30.8 = 14.4$ on $\Delta\text{df} = 2$ . A Poisson model has $\phi = 1$ , so $\Delta D/\phi = 14.4$ is the $\chi^2$ statistic. The critical value $\chi^2_{0.05, 2} = 5.99$ . Since $14.4 > 5.99$ , reject $H_0$ — territory significantly improves the model.