Exam ATPA

The Advanced Topics in Predictive Analytics course provides candidates with the ability to employ selected analytics techniques to solve business problems and effectively communicate the solution. After learning this material from e-Learning modules, candidates are given a take-home assessment requiring analysis of a data set in the context of a business problem and submission of a report. The report/assessment will be graded on a pre-set schedule.

About the exam

A thorough knowledge of probability (as covered in Exam P), mathematical statistics (as covered in VEE Mathematical Statistics), and selected models and methods for analyzing data (as covered in Exams SRM and PA) is assumed.

The learning objectives and outcomes provided on the following pages follow the four modules of the e-Learning support provided. The ranges of weights shown in the Learning Objectives below are intended to apply to the large majority of exams administered. On occasion, the weights of topics on an individual exam may fall outside the published range. Candidates should also recognize that tasks often cover multiple learning objectives, including some weight for communication and ethics in many tasks.

Learning Objectives

Ethical Foundations (5–10%) The Candidate will be able to work with data and models in an ethical and responsible manner.

1. Apply a general ethical framework for working with data and models.
2. Discuss and comply with relevant standards of practice.
3. Discuss and comply with relevant regulations that apply to working with data and models.

Working with Data (20–30%) The Candidate will be able to understand basic database calculations and manipulations and prepare data for predictive modeling applications.

1. Understand the basic structure of a data pipeline, including being able to perform the following tasks:
   • Evaluate the quality of appropriate data sources for a problem
   • Explain the difference between a database, data lake, and data warehouse
   • Describe how different data structures can be used in different analytical tasks.
2. Explain the basics concepts of database management, in particular, extract, transform, and load (ETL) operations. Demonstrate these skills by performing the following:
   • Extract data from various file structures
   • Subset, aggregate, summarize, and otherwise modify data for specific exploratory or modeling purposes
   • Create data sets as a final product of extracting and transforming data that can be used in a predictive model.
3. Assess the accuracy and quality of data.
   • Describe how data collection practices and assumptions affect data quality.
   • Validate the data for internal consistency.
4. Explain the terminology and structure of relational databases and be able to use common keys between collections of data to merge information from multiple sources.
5. Clean and organize data by performing each of the following:
   • Check for outliers, both univariate and multivariate
   • Handle missing data (including understanding the types of missing data) by selecting the appropriate action from deletion of the record, imputation, and adding a missing value flag.
6. Detect possible biases introduced when preparing data for a predictive model.

Advanced Predictive Analytics Models (30–45%) The Candidate will fit a variety of models and select one appropriate to the circumstances and intended use.

1. Explain the importance of model accuracy.
2. Explain, fit, evaluate, and make predictions with each of the following models:
   • Additive models
   • Linear mixed models
   • Neural networks.
3. Apply Bayesian techniques to predictive models.
4. Compare model results with those from linear and tree-based methods.
5. Explain the benefits of and demonstrate the combination of multiple models via stacking and blending.
6. Select and justify a modeling approach based on accuracy, explainability, stability, analytical effort, computational efficiency, and table importability, taking into account the business context of the problem.
7. Recognize and mitigate the effect of:
   • Starting with too many variables
   • Repeated use of train/test/validate sets
   • Model bias, including fairness concepts and proxy discrimination.

Model Explainability and Communication (25–35%) The Candidate will be able to effectively communicate the results of applying predictive analytics, including the relationship between model input and output, to solve a business problem.

1. Understand aspects of explainability, in particular:
   • The connection between ethics and explainability
   • Suitability, decomposability, algorithmic transparency, and post-hoc interpretability
   • The difference between explainability and interpretability
   • When a lack of explainability may be acceptable.
2. Communicate and justify a recommended analytics solution, including use as appropriate of:
   • Variable importance plots
   • Partial dependence plots
   • Individual conditional expectation plots
   • Shapley values
   • Lift and gain charts.
3. Explain why a model is predicting certain values for certain records.
4. Perform data and model governance and develop model documentation in an ethical context.
5. Communicate in a clear and straightforward manner using common language that is appropriate for the intended audience.
6. Structure a report in an effective manner while following standards of practice for actuarial communication.

Source Material

Source Material

• e-Learning Modules
  • All required resources are embedded in the e-Learning modules.