Human-Centered Algorithm Design for High Stakes Decision-Making in Public Services

点击翻译按钮获取中文摘要

Decades of neoliberal politics in North America centered on austerity and privatization have led to public sector agencies increasingly looking towards predictive algorithms and models built with artificial intelligence and machine learning technologies as a means to reduce costs, improve decision-making processes as well as provide greater efficiencies in public policy and social services delivery. Algorithms in the public sector have generally been adopted in the form of risk assessment algorithms with their primary purpose being the preemptive estimation of 'risk'. This has centered resources towards risk management based on individual client characteristics while driving attention away from structural and societal problems. Over the past two decades, several high-stakes decision-making domains such as the child-welfare system (CWS), criminal justice system, education, and healthcare have increasingly turned towards risk assessment algorithms. For instance, within CWS, there is growing public and media pressure because of the harm caused to children (especially minorities and Indigenous children given Canada's history of residential schools) who are removed from the care of their parents as well as where the system failed to remove and protect a child. However, the public sector poses its own challenges with respect to technical (quality of data), social and cultural (workers' interaction with algorithms), theoretical (what is risk assessment?), and societal (impact of algorithms on communities) implications of algorithmic decision-making. This research program will combine concepts from human-computer interaction (HCI), machine learning (ML) and participatory design (PD) to engage in the design and development of human-centered algorithms for high stakes decision making in the public sector. Particularly, the lens chosen will be through integrating participatory design strategies of relevant stakeholders with machine learning models that leverage historical, unstructured narratives to develop a Participatory Machine Learning (PML) framework. Further, this framework will advance the development of strength-based, holistic assessments that aim to produce positive outcomes for people as opposed to the current norm of narrow, deficit-based risk assessments that measure risk to the government. Finally, the PML framework will be validated in the public sector in two crucial areas of Canadian importance - child welfare and criminal justice. Through graduate student-led projects, this research program will advance insights into algorithmic fairness, bias and transparency issues in public services from a socio-technical perspective in order to train the next generation of researchers engaged in developing computational technologies for the social good. Ultimately the insights gleaned from this research program will be integrated into public sector programs for the purposes of better, high stakes algorithmic decision-making.