2.5 Predictive Risk Modelling In New Zealand, the development of a proposed predictive risk modelling (PRM) programme in the child protection sector may have significant implications for children’s privacy rights. The aim of the proposed programme, developed by the Ministry of Social Development (MSD), is to identify children at risk of maltreatment as they enter the public welfare system in order to target interventions and service delivery. PRM is generated from a large data set of public welfare and child protection services information. An algorithmic program is applied to the data to generate ‘risk’ scores for individuals. Service responses are then ascertained according to the risk score. As a PRM initiative requires agencies to share identifiable personal information without consent, it requires either an AISA83 or enabling provisions in primary legislation to legally override standard Privacy Act protections. Concerns have been raised about the ethics and human rights implications of PRM, including in relation to the security of information; unanticipated uses of information; stigmatisation of people identified as having high risk scores; systematic discrimination occurring as a result of the algorithmic techniques used to filter data; and transparency in relation to the data used to create algorithmic design.84 In order to ensure that privacy, human rights and ethical considerations are factored into the development and implementation of PRM, MSD is currently developing a Privacy, Human Rights and Ethics (PHRAE) Framework as a procedural safeguard. At the time of writing both the child protection PRM initiative and the PHRAE Framework are still under development and yet to be implemented. At this stage, it is understood that the PHRAE framework is intended to be a 83 See, for example, the 2015 Draft Youth Services AISA between MSD, the Ministry of Education and the Department of Corrections which proposed to enable personal data to be shared between those agencies for the purpose of identifying youth beneficiaries eligible to enrol in the MSDs Youth Service programme. The draft AISA is accessible at: https://www.msd.govt.nz/about-msd-and-our-work/work-programmes/ policy-development/youth-service-information-sharing-agreement-consultation/public-consultation-on-new-youth-service-information-sharing-agreement.html 84 See Keddell E,“The ethics of predictive risk modelling in Aotearoa/ New Zealand child welfare context: child abuse prevention or neo-liberal tool?”, 28 July 2014, available at https://ourarchive.otago.ac.nz/ bitstream/handle/10523/5666/PRMfinal3.pdf?sequence=1&isAllowed=y policy-level process that will be undertaken by Ministry officials and is not intended to be vested under any specific legislative or regulatory provision. It is notable that in 2016 the UN Committee on the Rights of the Child recommended that the New Zealand Government ensure “that the Privacy, Human Rights and Ethics framework governing predictive risk modelling takes in to consideration the potentially discriminatory impacts of this practice, is made public and is referenced in all relevant legislation.”85 The advent of this approach has coincided with extensive reforms to the legislation governing New Zealand’s child protection and youth justice jurisdictions. The Children, Young Person’s and their Families Act and Young Persons (Oranga Tamariki) Legislation Act has greatly expanded the powers of specified government agencies to share and use personal information held about children and their families, including enabling the creation of combined data sets.86 In doing so, legislation expressly provides for a principle that the well-being and best interests of a child will generally take precedence over any duty of confidentiality owed to the child or young person or a member of the child’s family.87 This is an example of primary legislation being used to over-ride the information privacy principles that otherwise would have applied under the Privacy Act in respect of sharing of personal information between agencies.88 Other PRM initiatives, such as one that was directed at identifying young people at risk of long-term benefit dependency,89 have relied upon AISAs. More generally, New Zealand academics at the University of Otago have commented on the use 85 UN Committee on the Rights of the Child, Concluding observations on the fifth periodic report of New Zealand, CRC/C/NZL/CO/5 (21 October 2016) paras 20(a) and 20(b) http://www.refworld.org/docid/587ceb574.html 86 Children, Young Persons and the Families (Oranga Tamariki) Legislation Act 2017, Clause 41 (ss 65A-66Q) http://www.legislation.govt.nz/ act/public/2017/0031/latest/DLM7064591.html (NOTE: at the date of writing it is still to commence). 87 Clause 41, new s 66(2). 88 See Office of the Privacy Commissioner submission on Oranga Tamariki Bill, https://privacy.org.nz/assets/Files/Reports-to-ParlGovt/Submission-on-the-CYPF-Oranga-Tamariki-Legislation-Bill.pdf 89 https://www.hrc.co.nz/files/7914/6483/4019/16g_Human_Rights_ Commission_feedback_on_draft_Youth_Service_AISA.pdf. 20