Judged by machines? The Impact of Artificial Intelligence Algorithms on the Legitimacy of the Court System

该研究项目解决了在跨国司法系统中迅速增加使用技术的问题，特别强调了英格兰和威尔士刑事司法系统的预期变化。这是一个具有全球意义的问题，在美国(1)、加拿大(2)和中国(3)都发生了显著的发展。英国政府的观点是，“司法系统必须拥抱新技术，抓住数字革命的机遇”(4)，包括人工智能(5)。一套雄心勃勃的改革已经开始实施，被认为比其他任何可比的司法管辖区都要广泛(6)，有人担心“改革计划正在推进……在没有主要立法的情况下”(7)。在本文的背景下，当前的技术指的是各种各样的措施：警务工具，如达勒姆警察局的shart（它帮助监管人员决定是否应该释放嫌疑人，将其留在牢房中，或使其有资格参加当地的康复计划），穿戴式视频，在线法庭(9)，人工智能和法律决策工具。在每一个例子中，计算机算法都被用来代替或指导人类的决策。在美国，当计算机辅助司法决策本身被裁定为商业秘密时，它引起了人们对偏见和缺乏透明度的担忧。英国独立警察投诉委员会已经认识到过度依赖随身携带的视频而排除其他来源的风险，并且观看可能会改变警察对事件的回忆（12）。鲍林等人用“自动司法”一词来描述刑事司法系统中人类代理的日益自动化和减少（13）。程序的紧凑加剧了向“精算”（14）或“行政”司法的转变，这带有固有的风险。2013年，祖伯夫认识到“所有可以自动化的东西都将被自动化”（15）。现在我们正进入这样一个阶段，技术的调查应用也进入了证据分类领域（16），模糊了存在了几个世纪的分界线，而巩固“机器偏见”的风险是真实存在的（17）。尼森鲍姆指出，人工智能与使用机器来完成手工任务有相似之处，并且可以识别出共同的影响：权力的转移、责任的丧失和潜在的危害（18）。通常会丧失功能完整性（19），为了实现第一个要素，决策者很可能会接受这一点。在司法系统内某些程序的自动化过程中，决策权从受到职业道德约束的受管制的警察和法律专业人员转移到受管理层委托的难以理解的算法上，这些算法在问题出现和目标改变时没有能力提出反对意见。“自动化偏见”（20）可能导致对通过通常不完美但功能正常的技术、算法或系统的行为得出的答案的毫无疑问的依赖，这些答案“倾向于效率而不是正当程序保障”（21）。正当程序变得几乎无法识别，决策责任变得分散，有效挑战决策的能力似乎可能受到阻碍（22）。

The research project addresses the rapidly increasing use oftechnology in justice systems transnationally, with specialemphasis on anticipated changes in the criminal justicesystem in England and Wales. This is a matter with globalsignificance, notable developments having occurred inUnited States (1), Canada (2), and China (3). The UKgovernment view is that "[t]he justice system must embracenew technologies and seize the opportunities of the digitalrevolution" (4), including artificial intelligence (5). Anambitious set of reforms has already been embarked on,considered broader than in any other comparable jurisdiction(6) and there are concerns that "the reform programme ismoving ahead [...] in the absence of primary legislation" (7).In the context of this paper, current technology refers to avariety of measures: policing tools such as Durham police'sHART (8) (which assists custody officers in deciding whether asuspect should be released, kept in the cell, or made eligiblefor a local rehabilitation programme), body worn video, onlinecourts (9), artificial intelligence and legal decision-makingtools. In each instance, computer algorithms are used tosubstitute or guide human decision-making. Where activelyemployed in the United States computer-aided judicialdecision-making has raised concerns (10) of bias and a lackof transparency when the algorithm itself was ruled a tradesecret (11). The UK Independent Police ComplaintsCommission has recognised a risk of over-reliance on bodyworn video to the exclusion other sources, and that viewingmight modify officers' recollections of events (12).The term 'automatic justice' has been used by Bowling et alto describe the increasing automation and reduction ofhuman agency in the criminal justice system (13). Acompaction of processes has exacerbated the move toward'actuarial' (14) or 'administrative' justice, with inherent risks.In 2013, Zuboff recognised 'everything that can beautomated, will be automated' (15). Now we are moving to astage where the investigative application of technology alsogoes to the realm of evidence sorting (16), blurring dividinglines which have existed for centuries, while the risk ofentrenching "machine bias" is a real one (17).Nissenbaum notes the parallels with the use of machines toperform manual tasks, and common effects can beidentified: a shift of power, a loss of accountability, and apotential for harm (18). There is often a loss of functionalintegrity (19), which may well be accepted by policy makersin order to achieve the first of those elements. In automatingcertain processes within the justice system, decision-making8 / 12power shifts from regulated police and legal professionalstrained in professional ethics to inscrutable algorithms,commissioned by management, which are not equipped toraise objections as problems emerge and objectives change.'Automation bias' (20) can lead to unquestioning reliance onanswers derived through the actions of an often imperfect,but functioning technology, algorithm or system, which'favours efficiency over due process safeguards' (21). As dueprocess changes to become almost unrecognisable, andresponsibility for decision-making becomes diffused, theability to effectively challenge decisions seems likely to beobstructed (22).