Human-Centered Algorithm Design for High Stakes Decision-Making in Public Services

数十年来，北美以紧缩和私有化为中心的新自由主义政治导致公共部门机构越来越多地将人工智能和机器学习技术构建的预测算法和模型作为降低成本、改善决策过程以及提高公共政策和社会服务效率的手段。公共部门的算法通常以风险评估算法的形式采用，其主要目的是对“风险”进行先发制人的估计。这使资源集中在基于个别客户特征的风险管理上，同时将注意力从结构和社会问题上转移开。在过去的二十年里，一些高风险的决策领域，如儿童福利系统（CWS），刑事司法系统，教育和医疗保健越来越多地转向风险评估算法。例如，在加拿大妇女服务中心内部，公众和媒体的压力越来越大，因为儿童（特别是少数民族和土著儿童，因为加拿大有寄宿学校的历史）被从父母的照顾中带走，以及该系统未能带走和保护儿童。然而，公共部门在技术（数据质量）、社会和文化（工人与算法的互动）、理论（什么是风险评估？）以及算法决策的社会影响（算法对社区的影响）。该研究计划将结合联合收割机的概念，从人机交互（HCI），机器学习（ML）和参与式设计（PD）从事设计和开发以人为本的算法，在公共部门的高风险决策。特别是，选择的透镜将通过整合相关利益相关者的参与式设计策略与机器学习模型，利用历史，非结构化的叙述来开发一个渐进式机器学习（PML）框架。此外，这一框架将推动基于实力的全面评估的发展，旨在为人们带来积极的结果，而不是目前衡量政府风险的狭隘的、基于赤字的风险评估。最后，将在加拿大两个重要的关键领域-儿童福利和刑事司法-在公共部门验证PML框架。通过研究生主导的项目，该研究计划将从社会技术的角度深入了解公共服务中的算法公平性，偏见和透明度问题，以培养下一代从事开发计算技术的研究人员。最终，从这项研究计划中收集到的见解将被整合到公共部门的计划中，以实现更好的高风险算法决策。

Decades of neoliberal politics in North America centered on austerity and privatization have led to public sector agencies increasingly looking towards predictive algorithms and models built with artificial intelligence and machine learning technologies as a means to reduce costs, improve decision-making processes as well as provide greater efficiencies in public policy and social services delivery. Algorithms in the public sector have generally been adopted in the form of risk assessment algorithms with their primary purpose being the preemptive estimation of 'risk'. This has centered resources towards risk management based on individual client characteristics while driving attention away from structural and societal problems. Over the past two decades, several high-stakes decision-making domains such as the child-welfare system (CWS), criminal justice system, education, and healthcare have increasingly turned towards risk assessment algorithms. For instance, within CWS, there is growing public and media pressure because of the harm caused to children (especially minorities and Indigenous children given Canada's history of residential schools) who are removed from the care of their parents as well as where the system failed to remove and protect a child. However, the public sector poses its own challenges with respect to technical (quality of data), social and cultural (workers' interaction with algorithms), theoretical (what is risk assessment?), and societal (impact of algorithms on communities) implications of algorithmic decision-making. This research program will combine concepts from human-computer interaction (HCI), machine learning (ML) and participatory design (PD) to engage in the design and development of human-centered algorithms for high stakes decision making in the public sector. Particularly, the lens chosen will be through integrating participatory design strategies of relevant stakeholders with machine learning models that leverage historical, unstructured narratives to develop a Participatory Machine Learning (PML) framework. Further, this framework will advance the development of strength-based, holistic assessments that aim to produce positive outcomes for people as opposed to the current norm of narrow, deficit-based risk assessments that measure risk to the government. Finally, the PML framework will be validated in the public sector in two crucial areas of Canadian importance - child welfare and criminal justice. Through graduate student-led projects, this research program will advance insights into algorithmic fairness, bias and transparency issues in public services from a socio-technical perspective in order to train the next generation of researchers engaged in developing computational technologies for the social good. Ultimately the insights gleaned from this research program will be integrated into public sector programs for the purposes of better, high stakes algorithmic decision-making.