PLEAD: Provenance-driven and Legally-grounded Explanations for Automated Decisions

算法和人工智能如今在许多技术系统中发挥着关键作用，这些系统控制或影响着我们生活的各个方面。他们每天根据交通状况优化我们的驾驶路线；他们决定我们的抵押贷款申请是否得到批准；他们甚至会给我们推荐潜在的生活伴侣。他们默默地在幕后工作，不被我们注意到，直到他们不被我们注意到。当我们的信用卡申请在两秒钟内被批准时，我们中很少有人会想太多。只有当它被拒绝时，我们才开始质疑这个决定。大多数时候，我们得到的答案是不令人满意的，如果我们得到任何答案的话。这种不透明的自动决策在日常生活中的普及，推动了公众对算法问责制的要求，即有义务解释和证明自动决策的合理性。主要的担忧是，这些算法（实际上是黑箱）不应该接受我们的数据，并以我们不理解的方式做出影响我们的决定。出于这个原因，《通用数据保护条例》（General Data Protection Regulation）要求我们作为数据主体，被提供“有关所涉及的逻辑、以及此类处理的重要性和预期后果的有意义的信息”。同样，根据金融服务法规，消费者在接受金融服务时应得到公平对待，根据数据保护、平等和人权法，算法应不受歧视。然而，由于法律法规没有规定如何满足这些要求，企业不得不自己解释这些要求，采用各种手段，包括报告，互动网站，甚至专门的呼叫中心，向客户提供解释。在这种背景下，出处，特别是它的标准证明，描述了一条信息或数据是如何产生的，以及是什么影响了它的产生。在记录的来源轨迹中，我们可以追溯自动决策以提供一些问题的答案，例如使用哪些数据来支持决策，谁或哪个组织对数据负责，还有谁可能受到影响。虽然来源信息在结构上很简单，但是从自动化系统中捕获的来源对于人类的消费来说往往是压倒性的。此外，简单地向一个人提供出处并不一定构成一种解释。需要对其进行总结并提取其精华，以便能够构建针对特定监管目的的解释。我们今天还不知道如何做到这一点。该公司汇集了一个由技术专家、法律专家、商业公司和公共组织组成的跨学科团队，研究数据来源如何有助于解释自动决策背后的逻辑，从而造福数据主体，并帮助数据控制者证明自己遵守了法律。特别是，我们将识别与算法决策的目的相关的各种有意义的解释，并根据适用于英国企业的与数据保护、歧视和金融服务相关的法律要求对其进行分类。在此基础上，我们将构思解释生成算法，处理、总结和抽象由自动化决策管道记录的来源。将为数据控制器创建一个解释助手工具，为其应用程序提供基于来源的解释功能。在整个项目中，我们将通过访谈和用户研究与合作伙伴、数据主体、数据控制者和监管机构进行接触，以确保解释符合目的和意义。因此，以来源为导向和法律依据的解释将使数据主体信任自动决策，并使数据控制者确保遵守对其组织的法律要求。

Algorithms and Artificial Intelligence play a key role nowadays in many technological systems that control or affect various aspects of our lives. They optimise our driving routes every day according to traffic conditions; they decide whether our mortgage applications get approved; they even recommend us potential life partners. They work silently behind the scene without much of our notice, until they do not. Few of us would probably think much about it when our credit card application is approved in two seconds. Only when it is rejected, do we start to question the decision. Most of the time, the answers we get are not satisfactory, if we get any at all. The spread of such opaque automated decision-making in daily life has been driving the public demand for algorithmic accountability - the obligation to explain and justify automated decisions. The main concern is that it is not right for those algorithms, effectively black boxes, to take in our data and to make decisions affecting us in ways we do not understand. For this reason, the General Data Protection Regulation requires that we, as data subjects, be provided with "meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing." Likewise, consumers should be treated fairly when receiving financial services as per financial services regulations and algorithms should be free of discrimination as per data protection, equality and human rights laws. However, as laws and regulations do not prescribe how to meet such requirements, businesses are left with having to interpret those themselves, employing a variety of means, including reports, interactive websites, or even dedicated call centres, to provide explanations to their customers.Against this background, provenance, and specifically its standard PROV, describes how a piece of information or data was created and what influenced its production. Within recorded provenance trails, we can retrace automated decisions to provide answers to some questions, such as what data were used to support a decision, who or what organisation was responsible for the data, who else might have been impacted. While provenance information is structurally simple, provenance captured from automated systems, however, tends to be overwhelming for human consumption. In addition, simply making provenance available to a person does not necessarily constitute an explanation. It would need to be summarised and its essence extracted to be able to construct an explanation addressing a specific regulatory purpose. How we do this is unknown today.PLEAD brings together an interdisciplinary team of technologists, legal experts, commercial companies and public organisations to investigate how provenance can help explain the logic that underlies automated decision-making to the benefit of data subjects as well as help data controllers to demonstrate compliance with the law. In particular, we will identify various types of meaningful explanations for algorithmic decisions in relation to their purposes, categorise them against the legal requirements applicable to UK businesses relating to data protection, discrimination and financial services. Building on those, we will conceive explanation-generating algorithms that process, summarise and abstract provenance logged by automated decision-making pipelines. An Explanation Assistant tool will be created for data controllers to provision their applications with provenance-based explanations capabilities. Throughout the project, we will engage with partners, data subjects, data controllers, and regulators via interviews and user studies to ensure the explanations are fit for purpose and meaningful. As a result, explanations that are provenance-driven and legally-grounded will allow data subjects to place their trust in automated decisions, and will allow data controllers to ensure compliance with legal requirements placed on their organisations.