Exam 8 Practice Question 6
Study guide: Exam 8
A modeler adds vehicle make and model, with about 12,000 levels, to a physical damage frequency GLM. For each make and model, a credibility-shrunk mean frequency is computed from the whole training set and added as a single numeric feature (target encoding). In 5-fold cross-validation the new model looks far better than the old one, and the encoded feature gets a large coefficient. On a later holdout year it does worse than the old model without the feature. Every make and model in the holdout year also appears in the training data.
What is the most likely cause, and the fix?
- A) Make and model has too many levels to carry any signal, so the feature should be dropped
- B) The shrinkage should use the Bühlmann ratio of variances rather than a
\lambdachosen by cross-validation - C) Make and model should be one-hot encoded in an unpenalized GLM, so each level gets its own coefficient
- D) Each record's encoding includes its own claims, so each fold should be encoded from the other folds only
- E) New makes and models in the holdout were encoded as zero, so they should get the overall mean instead
Answer this question and see the worked solution