4 | Artificial Intelligence and discrimination 4.3 Mitigating against algorithmic bias and discrimination The effects of algorithmic bias may be avoided through rigorous design, testing and monitoring of an AI system. AI systems should generally be designed, where possible, to avoid discriminatory outcomes. For insurers, AI systems may necessarily include some forms of discrimination, so should be designed to avoid unlawful discrimination. They should be monitored and tested throughout their lifecycle to ensure they continue to do so, and any identified problems are appropriately addressed. In this way, it is important that relevant technical staff and management have training in relation to discrimination law to ensure that problems can be identified and addressed. In addition to the above, there are various mitigation strategies that can be employed in relation to the data and models used by AIsystems to address algorithmic bias and prevent unlawful discriminatory outcomes. This section provides some examples of such strategies, but is not intended to be an exhaustive list. This is an emerging field with conflicting views (particularly regarding measuring fairness, as discussed above) and, at times, mutually contradictory proposals. Insurers should keep up to date with the emerging literature, and ensure they carefully consider their specific context in justifying any approach taken. Insurers should record any steps taken to mitigate against unlawful discrimination, and the reasons for those decisions. This information may assist a court in understanding the reasons for any unfavourable treatment, determining whether any indirect discrimination was reasonable in the circumstances, or determining whether any discrimination based on data was reasonable, and if a relevant exemption applies. 22 (a) Data Insurance pricing and underwriting decisions are driven by data. This is recognised by the insurance exemptions under the ADA, DDA and SDA, which, in certain circumstances, allow an insurer to discriminate if it is based on actuarial or statistical data. Data can be internal or external to the insurer. The term ‘data’ covers raw data but can also cover outputs from other models and analysis undertaken (including, for example, government studies and academic studies) that are used as inputs into AI-informed decision making systems, as well as any resulting interpretation or professional judgement of the data and analysis. It is important to understand the potential issues of a data set used to train an AI system. Steps may need to be taken to address these issues and avoid discriminatory outcomes, such as acquiring more data (particularly from underrepresented segments of the population), preprocessing the data, not including data relating to protected attributes within customer decisions to remove direct discrimination, or applying suitable mitigation strategies within algorithm or model construction to remove indirect discrimination. (i) Acquire more representative, more appropriate, or additional data The quality of the data that is used to train an AIinformed decision making system will affect the quality of the decisions. This is sometimes called the ‘garbage in, garbage out’ problem.