Human-Centered Algorithm Design for High Stakes Decision-Making in Public Services

北美几十年来以紧缩和私有化为中心的新自由主义政治导致公共部门机构越来越多地寻求使用人工智能和机器学习技术建立的预测算法和模型，以此作为降低成本、改进决策过程以及提高公共政策和社会服务提供效率的手段。公共部门的算法通常以风险评估算法的形式被采用，其主要目的是先发制人地估计“风险”。这将资源集中到基于个人客户特征的风险管理上，同时将注意力从结构和社会问题上移开。在过去的二十年里，儿童福利系统(CWS)、刑事司法系统、教育和医疗保健等几个高风险决策领域越来越多地转向风险评估算法。例如，在CWS内部，由于儿童(特别是少数族裔和土著儿童，鉴于加拿大的寄宿学校历史)受到伤害，被从父母的照料中带走，以及系统未能带走和保护儿童，在CWS内部，公众和媒体的压力越来越大。然而，公共部门在算法决策的技术(数据质量)、社会和文化(工作人员与算法的交互)、理论(什么是风险评估？)和社会(算法对社区的影响)方面提出了自己的挑战。这项研究计划将结合人机交互(HCI)、机器学习(ML)和参与式设计(PD)的概念，致力于设计和开发以人为中心的算法，用于公共部门的高风险决策。特别是，选择的镜头将通过将相关利益相关者的参与式设计战略与机器学习模型相结合，这些模型利用历史的、非结构化的叙述来开发参与式机器学习(PML)框架。此外，这一框架将推动基于实力的全面评估的发展，旨在为人民带来积极的结果，而不是目前衡量政府风险的狭隘、基于赤字的风险评估的规范。最后，PML框架将在加拿大重要的两个关键领域--儿童福利和刑事司法--在公共部门得到验证。通过研究生主导的项目，这项研究计划将从社会技术角度推动对公共服务中算法公平、偏见和透明度问题的洞察，以培训致力于为社会公益开发计算技术的下一代研究人员。最终，从这项研究计划中收集到的见解将被整合到公共部门计划中，以实现更好、更高风险的算法决策。

Decades of neoliberal politics in North America centered on austerity and privatization have led to public sector agencies increasingly looking towards predictive algorithms and models built with artificial intelligence and machine learning technologies as a means to reduce costs, improve decision-making processes as well as provide greater efficiencies in public policy and social services delivery. Algorithms in the public sector have generally been adopted in the form of risk assessment algorithms with their primary purpose being the preemptive estimation of 'risk'. This has centered resources towards risk management based on individual client characteristics while driving attention away from structural and societal problems. Over the past two decades, several high-stakes decision-making domains such as the child-welfare system (CWS), criminal justice system, education, and healthcare have increasingly turned towards risk assessment algorithms. For instance, within CWS, there is growing public and media pressure because of the harm caused to children (especially minorities and Indigenous children given Canada's history of residential schools) who are removed from the care of their parents as well as where the system failed to remove and protect a child. However, the public sector poses its own challenges with respect to technical (quality of data), social and cultural (workers' interaction with algorithms), theoretical (what is risk assessment?), and societal (impact of algorithms on communities) implications of algorithmic decision-making. This research program will combine concepts from human-computer interaction (HCI), machine learning (ML) and participatory design (PD) to engage in the design and development of human-centered algorithms for high stakes decision making in the public sector. Particularly, the lens chosen will be through integrating participatory design strategies of relevant stakeholders with machine learning models that leverage historical, unstructured narratives to develop a Participatory Machine Learning (PML) framework. Further, this framework will advance the development of strength-based, holistic assessments that aim to produce positive outcomes for people as opposed to the current norm of narrow, deficit-based risk assessments that measure risk to the government. Finally, the PML framework will be validated in the public sector in two crucial areas of Canadian importance - child welfare and criminal justice. Through graduate student-led projects, this research program will advance insights into algorithmic fairness, bias and transparency issues in public services from a socio-technical perspective in order to train the next generation of researchers engaged in developing computational technologies for the social good. Ultimately the insights gleaned from this research program will be integrated into public sector programs for the purposes of better, high stakes algorithmic decision-making.