Actuarial Notes

Data Documentation

Study guide: Exam DISC-DA

Data Documentation is the written record of what a data set contains and how it was made: a definition, type, unit and valid range for every field, where each field came from, and every transformation applied on the way. It lets someone who did not build the data, such as a modeler, a reviewer or a regulator, use it correctly and reproduce it, and it is the first defence of data quality in an insurer's modelling project.

E_{[a,b]} = \sum_i u_i \times \frac{\left|[s_i, e_i] \cap [a, b]\right|}{e_i - s_i}

\text{field} = T_k \circ \cdots \circ T_2 \circ T_1(\text{source})

  • Symbols: The first block is the kind of precise definition a data dictionary should hold, here for earned exposure over a reporting period [a, b]. Policy i runs from s_i to e_i and carries written exposure u_i in stated units, such as car-years, and the fraction is the share of its term that falls inside the period. The second block is a field's lineage: the ordered transformations T_1, \dots, T_k that turn a source field into the field as delivered.
  • Data dictionary: A data dictionary lists every field with its name, plain-language definition, data type and format, unit, allowed values or range, how missing values are coded, the source system, and the owner who can answer questions about it. A coded field also needs its code list, because 03 means nothing until the dictionary says which territory or coverage it stands for.
  • Metadata: Metadata is data about the data set as a whole. It records when the data were extracted and as of what valuation date, the number of records and the control totals, the filters applied, the version, and who prepared it. The control totals are what later validation reconciles against.
  • Lineage: Lineage traces each field back through the pipeline to its source: which tables were joined, which records were excluded, how amounts were converted, and which rules derived a new field. When a number looks wrong, lineage is what lets the analyst find the step that changed it.
  • Transformations and assumptions: A derived field is documented with its formula and the judgments behind it. Examples are whether losses include allocated loss adjustment expense, whether salvage and subrogation are netted, how a policy cancelled mid-term is earned, and how outliers were capped.
  • Why it matters: The same word often means different things in different departments. Underwriting may count exposure in policies, ratemaking in car-years, and finance may book premium net of ceded reinsurance. Documentation fixes one meaning per field, so that two analysts who join the same tables get the same answer.
  • Standards: ASOP No. 23 asks the actuary to make a reasonable effort to learn the definition of each data element and to disclose the data's limitations, and model governance guidance expects the data behind a model to be assessed and documented: U.S. bank supervisors' model risk guidance (SR 11-7) asks for documentation detailed enough that someone unfamiliar with the model can understand how it operates, its limitations and its key assumptions, and ASOP No. 56 asks for documentation from which another qualified actuary could assess the reasonableness of the work.

A Data Dictionary Entry for Earned Exposure An auto insurer's modelling data set has a field earned_exposure_cy, which holds the exposure each policy earned in calendar year 2024. Two policies illustrate it. Policy A insures 2 cars for 12 months from October 1, 2024 to October 1, 2025. Policy B insures 1 car for 6 months from August 15, 2024 to February 15, 2025. Write the dictionary entry for the field and compute its value for each policy, counting days.

Answer A useful entry for the field reads as follows.

  • Definition: The car-years of coverage provided by the policy term within calendar year 2024, earned evenly by day.
  • Unit and type: The unit is car-years, where 1 car insured for 365 days is 1.0000, and the type is a decimal with 4 places.
  • Range: The value lies between 0 and the term's written car-years, and a negative value is an error.
  • Source and lineage: It comes from policy_term (effective date, expiry date and vehicle count) and is derived as written car-years times the share of the term's days inside 2024. A cancelled term uses its cancellation date as e_i.
  • Owner and as-of date: The pricing data team owns it, and it was extracted as of March 31, 2025.

Policy A writes 2 car-years, and 92 of its 365 days fall in 2024. Policy B writes 0.5 car-years over 184 days, and 139 of them fall in 2024.

\begin{align*}
E_A &= 2 \times \frac{92}{365} \\
&= 0.5041 \\
E_B &= 0.5 \times \frac{139}{184} \\
&= 0.3777
\end{align*}

Together the two policies earned 0.8818 car-years in 2024. The definition matters for Policy B: counted in policy terms instead of car-years, it would show 0.7554, twice its car-years.

What Goes Wrong Without a Definition A pricing analyst receives a file of six-month auto policies with a field named exposure and no data dictionary. In the year, 2,000 policy terms were fully earned and incurred losses were $1,200,000. The rate manual quotes premiums per car-year. The analyst assumes exposure counts car-years, but it actually counts policy terms. What pure premium does the analyst compute, and what should it be?

Answer Each six-month term is half a car-year, so the 2,000 terms are 1,000 car-years.

\begin{align*}
\text{PP}_{\text{analyst}} &= \frac{1{,}200{,}000}{2{,}000} \\
&= 600 \\
\text{PP}_{\text{correct}} &= \frac{1{,}200{,}000}{1{,}000} \\
&= 1{,}200
\end{align*}

The analyst's $600 is a pure premium per six-month term, but it would be read as $600 per car-year, so the indicated annual rate is half what the losses support. Nothing in the data looks wrong, because every value is valid. Only a dictionary entry stating the unit, or a reconciliation of exposure to the policy count, would catch it.

How it appears on the exam

  • DISC-DA: Questions describe a modelling project and ask which element of documentation is missing or what a data dictionary should record for a field. Others ask why lineage matters when a result cannot be reproduced. Expect application questions that pair documentation with data quality principles and data classifications from the same topic.
  • Scenario questions: A common shape gives two analysts who reach different totals from the same source and asks for the cause. The answer is usually an undocumented definition, filter or transformation, such as a field counting policies in one extract and car-years in the other.

Exam traps

  • A field name is not a definition: A name like exposure or losses leaves open the unit, the basis and what is included. The documentation has to say whether losses are paid or incurred, whether they include expenses, and whether they are gross or net of recoveries.
  • Metadata versus the data dictionary: The data dictionary describes each field, while metadata describes the data set as a whole, such as its extract date, record count, filters and version. A question about control totals or the as-of date is about metadata.
  • Lineage is about transformations, not storage: Lineage records how a field was derived and which records were dropped on the way, not just which database it sits in. Without it, an excluded line of business or a changed rule is invisible.
  • Documentation does not validate the data: Documenting a field correctly does not make its values right. The values still need the range, consistency and reconciliation checks of data validation.