The Colorado Division of Insurance Continues to Navigate Life Insurer’s Use of ECDIS and Algorithms
- Industry Survey and Data Call
During the July 8 stakeholder meeting, the CO DOI announced that it will issue a survey and data call to a sampling of life insurance companies. The CO DOI proposed that the top 10 carriers with the largest number of covered lives would be asked to complete the survey and data call. The CO DOI requested comments on their proposed survey and data call by July 22, 2022, and expects to issue the survey and data call in mid-August. The CO DOI indicated that it would hold another stakeholder meeting before the survey and data call are issued but also stated that it might be possible to address the comments without needing a meeting.
The CO DOI believes that the survey and data call will provide it a bearing on the industry’s:
- Current practices on the use of ECDIS and algorithms.
- Views on the different components of SB-169.
The proposed survey will collect information regarding a company’s practices with respect to governance and testing processes for unfair discrimination. The survey questions seek to determine what insurers are doing to assess whether their use of algorithms or predictive models using ECDIS results in a disproportionately negative outcome for protected classes that exceeds reasonable correlations to the underlying insurance practices. In particular, the survey is seeking information with respect to four categories, as follows:
- Processes and procedures around unfair discrimination, including (i) whether there are any specific external data sources or types of external data the insurer does not use, as a matter of internal company policy, and (ii) which of the insurer’s external data sources, predictive models, or insurance practices are currently tested for unfair discrimination.
- Assessments of external data sources – i.e., inputs to algorithms or predictive models, including how the insurer decides whether a given ECDIS needs to be tested, what bias is being tested, what testing methodology is employed, and what analysis is employed to assess a given ECDIS.
- Assessment of the entirety of predictive models, including what outcomes are considered in testing predictive models, what bias is being tested, what testing methodology is employed to determine race or other demographic information, what analysis is employed to assess outcomes such as whether there is a disproportionately negative outcome for certain demographic groups, and what measures are taken to mitigate problems once they’ve been discovered.
- Documentation of actual instances of the governance and testing procedures, which seeks actual documentation of the governance and testing procedures that an insurer has conducted and information on whether there was a finding, or no finding, of unfair discrimination.
In addition to the proposed survey, the CO DOI proposes to launch a data call to determine if the insurers’ data would be seaworthy for stress testing to ascertain whether unfair discrimination exists in their use of ECDIS. The proposed data call seeks information on:
- Application data for products that use ECDIS in one or more algorithms/predictive models
The proposed data call seeks specific information regarding applications received during a specific time frame with a minimum of 10,000 applications for life insurance products that use models incorporating ECDIS in the underwriting process.
- List of products that do NOT use ECDIS in at least one algorithm/predictive model, and a brief explanation of what data/information is used and why it is outside of the scope of the law.
Birny Birnbaum suggested the CO DOI consider trimming the sails by limiting the survey and data call to term life or whole life since those are the products being sold to underserved markets at the highest risk of unfair discrimination.
The CO DOI emphasized that it would not use the information from the survey or data call to sink an insurer in any enforcement action.
- ACLI Proposed Algorithmic Accountability Regulation
As the stakeholder meeting got underway, Jillian Froment, former Ohio Insurance Director, presented on the principle-based ACLI Proposed Algorithmic Accountability Regulation (ACLI Proposal). The ACLI Proposal adopts a risk management framework that addresses unfair discrimination in underwriting based on race. The risk management framework would require insurers to:
(a) develop and maintain an internal governance process regarding the use of ECDIS in underwriting to mitigate the risk of unfair discrimination;
(b) prepare a report documenting the insurer’s application of its internal governance process regarding the use of ECDIS in underwriting to mitigate the risk of unfair discrimination;
(c) perform and maintain Impact Assessments (IAs) to determine if the use of ECDIS in an algorithm or predictive model in underwriting results in unfair discrimination based on race; and
(d) prepare an IA report documenting the insurer’s analysis of its use of ECDIS in its algorithms and predictive models under a two-part structure.
Under the principles-based approach, each insurer would determine the most appropriate heading and course for testing to determine whether the insurer's use of ECDIS results in unfair discrimination based on race that exceeds the reasonable correlation to the underlying insurance practice. Consumer representative Birny Birnbaum criticized this approach, stating it would result in the CO DOI having to review multiple methods of testing as they would vary by insurer. Colorado Commissioner Conway expressed concern that the ACLI Proposal sailed a bit out of the shipping lanes as he did not agree that insurers should determine the appropriate method or threshold for testing for unfair discrimination. He offered that the Colorado regulation would chart the course for the requisite testing.
The ACLI Proposal’s two-part reporting structure would steer insurers to first document their internal governance process, testing, methodologies, and outcomes (Part 1) and then summarize the prior documentation and provide an attestation that they have completed the testing in compliance with the regulation (Part 2). The ACLI Proposal contemplates that an insurer would file both Part 1 and Part 2 in the early years, and if the insurer's use of ECDIS has not changed course from year to year, the insurer could simply file Part 2 annually and make Part 1 available to the CO DOI upon request. The Commissioner expressed concern that given the speed at which the industry is innovating and evolving in its use of ECDIS, that there would always be changes to Part 1.
The Commissioner’s comments on the ACLI Proposal, proposed survey, and data call make clear that the CO DOI has a different tact in mind than the ACLI Proposal. We will keep watching the direction of the wind to see how insurers should hoist their sails.
