Actuarial Notes

Regulatory Review of Predictive Models White Paper

By Casualty Actuarial and Statistical (C) Task Force · National Association of Insurance Commissioners · 2020

Study guide: Exam 6U

A white paper of the NAIC's Casualty Actuarial and Statistical (C) Task Force that identifies best practices for state insurance regulators reviewing the predictive models behind insurers' rating plans, with state guidance for rate filings based on complex predictive models. The Task Force adopted it on September 15, 2020, the Property and Casualty Insurance (C) Committee on December 8, 2020, and the Executive (EX) Committee and Plenary on April 14, 2021.

On the syllabus

  • Exam 6U — objective A5; Sections I, IV and VII (the white paper dated December 2020).

I Introduction

  • Predictive analytics and big data have significant potential benefits to consumers and insurers, but techniques are evolving rapidly and leave many state regulators without the tools to review insurers' use of predictive models.
  • The Task Force was charged to draft changes to the Product Filing Review Handbook to include best practices for review of predictive models filed to justify rates, and to draft state guidance for rate filings based on complex predictive models.
  • The document is guidance for state regulators; nothing in it is intended to change the applicable legal and regulatory standards for approval of rating plans.

II What is a "Best Practice?"

  • Key Regulatory Principles
      1. State insurance regulators will maintain their current rate regulatory authority and autonomy.
      1. State insurance regulators will be able to share information to aid companies in getting insurance products to market more quickly across the states.
      1. State insurance regulators will share expertise and discuss technical issues regarding predictive models to make the review process in any state more effective and efficient.
      1. State insurance regulators will maintain confidentiality, in accordance with state law, regarding predictive models.

III Some Issues in Reviewing Today's Predictive Models

  • Generalized Linear Models
  • Credibility of GLM Output

IV Do Regulators Need Best Practices to Review Predictive Models?

  • Best practices are of value to regulators and insurance companies: they provide a baseline of analysis without creating unfair or inordinate barriers for insurers, and are not intended to create standards for filings that include predictive models.
  • Best practices provide guidance to state regulators in their authoritative role over rating plans, a framework for sharing knowledge and resources, and aid in training regulators new to reviewing predictive models.

V Scope

VI Confidentiality

VII Best Practices for the Regulatory Review of Predictive Models

  • The regulator's review of predictive models should:
      1. Ensure that the selected rating factors, based on the model or other analysis, produce rates that are not excessive, inadequate, or unfairly discriminatory.
      1. Obtain a clear understanding of the data used to build and validate the model, and thoroughly review all aspects of the model, including assumptions, adjustments, variables, sub-models used as input, and resulting output.
      1. Evaluate how the model interacts with and improves the rating plan.
      1. Enable competition and innovation to promote the growth, financial stability, and efficiency of the insurance marketplace.

VIII Proposed Changes to the Product Filing Review Handbook

  • Generalized Linear Models
  • What is a "Best Practice"?
  • Best Practices for the Regulatory Review of Predictive Models
  • Confidentiality

IX Proposed State Guidance

X Other Considerations

Appendix A Best Practices Development

    1. Define Scope
    1. Identify Top Performers
    1. Analyze Best Practices
    1. Adapt
    1. Implement and Evaluate

Appendix B Information Elements and Guidance for a Regulator to Meet Best Practices' Objectives (When Reviewing GLMs)

  • A. Selecting Model Input
  • B. Building the Model
  • C. The Filed Rating Plan
  • Mapping Best Practices to Information Elements and Information Elements to Best Practices

Appendix C Glossary of Terms

Appendix D Sample Rate-Disruption Template

Sources