4 | Artificial Intelligence and discrimination Insurers may need technical or actuarial advice regarding the availability, appropriateness and application of such techniques, taking into account the context of the model, and its materiality and effect on the outcomes of the model or AI system. For example, grouping middle aged people into cohorts of 5 years for travel insurance may be noncontroversial. However, for a product like motor insurance, where risk materially changes from ages 18 to 22, a finer level of grouping may be more appropriate. If the insurer is intending to rely on preprocessed data for an exemption under the ADA, DDA, or SDA, it must be reasonable for it to rely on the data as processed. Insurers should ensure that all processing steps are recorded and appropriately justified in the event that it needs to explain why it was reasonable for it to rely on the preprocessed data. (iii) Data relating to protected attributes The inclusion and selection of data relating to protected attributes is another important mitigation strategy. Noting that where the insurer satisfies the requirements of an applicable data exemption, it can include data relating to age, sex, or disability and discriminate on the basis of that data. However, it may be necessary for the insurer to remove data relating to other protected attributes. For example, whilst an insurer can discriminate based on data relating to age when calculating life insurance premiums, there is no exemption under the RDA that allows discrimination based on data relating to race. Preprocessing the data can also be used to edit features in the data set to mask or remove some information relating to protected attributes 24 before it is used to train or score an algorithm. For example, to remove direct discrimination, an individual’s sex could be hidden before it is used in model scoring, where the model is applied to the dataset. Alternatively, to protect against direct discrimination, an insurer might not collect or include any data relating to a protected attribute in its model or the decision-making process. To remove indirect discrimination, other mitigating strategies can then be used. Ultimately, an AI system must be tested to determine whether the strategy employed actually avoids discriminatory outcomes. For example, simply hiding protected attributes from the data set may not be sufficient to prevent discrimination if (as is likely in a sufficiently rich or granular dataset) other pieces of information in the dataset act as proxy variables for that protected attribute. Where a proxy variable is known, it may be possible to remove the discriminatory effect via statistical techniques. However, again, this is an area of evolving practice, with conflicting views in the academic research, so careful consideration should be given to the techniques employed, and why they are justified. (b) Models In simple terms, a model is a tool or algorithm that utilises a set of data to recognise patterns and make decisions. It may be possible to design or adjust models to avoid discriminatory outcomes, such as by adjusting the design or parameters of the model, or by changing its complexity. Testing and ongoing monitoring of the model are important mitigation strategies against discrimination. To do so effectively, there may need to be an understanding of the reasons for the model’s outputs, as well as human oversight of the model and its outputs.