Making Systems Answer: Dialogical Design as a Bridge for Responsibility Gaps in Trustworthy Autonomous Systems

随着计算机系统变得越来越自主--能够独立驾驶车辆，检测欺诈性银行交易，或者读取和诊断我们的医疗扫描--人类能够自信地评估和确保它们的可信度是至关重要的。我们的项目开发了一种新颖的、以人为中心的方法，以克服这方面的一个主要障碍，即责任差距。当我们无法确定对具有高道德风险的行动负有道德责任的人时，就会出现责任空白，要么是因为不清楚谁是幕后黑手，要么是因为代理人不符合道德责任的条件；例如，如果行为不是自愿的，或者代理人不知道。责任差距是一个问题，因为让别人对自己的所作所为负责是我们维持社会信任的方式。自治系统造成了新的责任缺口。他们在医疗和金融等高风险领域运作，但他们的行为可能不受道德责任人的控制，或者由于驱动这些行为的复杂的黑箱算法，人类可能无法完全理解或预测这些行为。为了让这样的系统值得信赖，我们需要找到一种方法来弥合这些差距。我们的项目利用哲学、认知科学、法律和人工智能方面的研究，为自主系统开发人员、用户和监管机构开发新的方法，以弥合责任差距--通过提高系统交付责任的一个重要且未被充分研究的部分的能力，即可应答性。当我们说某人对一种行为负有责任时，这是谈论他们的责任的一种方式。但问责性不是要责备某人，而是要向受我们行为影响的人提供他们需要或期望的答案。负责任的人以许多不同的方式对行为负责；他们可以解释、辩解、重新考虑、道歉、提供补偿、做出改变或采取未来的预防措施。与计算中的可解释性或法律上的责任性相比，可问责性包含了一套更丰富的责任实践。通常，为我们的行为负责的行为本身就会改进我们，帮助我们在未来变得更负责任和更值得信赖。这就是为什么问责性是弥合责任差距的关键。这不是关于我们指定谁为‘责任人’(在自治系统中更难识别)，而是我们对让系统负责的人欠了什么。如果系统作为一个整体(机器+人)能够更好地给出应得的答案，系统仍然可以满足现在和未来对他人的责任。因此，问责性是一种系统执行责任的能力，可以弥补责任差距。我们的目标是提供理论和经验证据以及计算技术，以演示如何使自治系统(包括由开发人员、所有者、用户等组成的更广泛的“系统”)能够提供人们从值得信赖的代理那里寻求的答案。我们的第一个工作流程建立了理论和概念框架，使系统开发人员、用户和监管机构能够更好地理解和执行可回答问题。第二个工作流程以以人为本、以证据为导向的方法为基础，让自主系统的各种公众、用户、受益者和监管者参与研究。焦点小组、研讨会和访谈将用于讨论卫生、金融和政府领域的案例和情景，揭示人们对在这些领域运行的值得信赖的系统所期望的答案。最后，我们的第三个工作流程开发了新的计算人工智能技术，通过与用户和监管机构更具对话性和响应性的界面来提高自主系统的可应答性。我们的研究成果和活动将产生学术、行业和面向公众的资源的组合，用于设计、部署和管理更负责任的自主系统。

As computing systems become increasingly autonomous--able to independently pilot vehicles, detect fraudulent banking transactions, or read and diagnose our medical scans--it is vital that humans can confidently assess and ensure their trustworthiness. Our project develops a novel, people-centred approach to overcoming a major obstacle to this, known as responsibility gaps. Responsibility gaps occur when we cannot identify a person who is morally responsible for an action with high moral stakes, either because it is unclear who was behind the act, or because the agent does not meet the conditions for moral responsibility; for example, if the act was not voluntary, or if the agent was not aware of it. Responsibility gaps are a problem because holding others responsible for what they do is how we maintain social trust. Autonomous systems create new responsibility gaps. They operate in high-stakes areas such as health and finance, but their actions may not be under the control of a morally responsible person, or may not be fully understandable or predictable by humans due to complex 'black-box' algorithms driving these actions. To make such systems trustworthy, we need to find a way of bridging these gaps. Our project draws upon research in philosophy, cognitive science, law and AI to develop new ways for autonomous system developers, users and regulators to bridge responsibility gaps-by boosting the ability of systems to deliver a vital and understudied component of responsibility, namely answerability. When we say someone is 'answerable' for an act, it is a way of talking about their responsibility. But answerability is not about having someone to blame; it is about supplying people who are affected by our actions with the answers they need or expect. Responsible humans answer for actions in many different ways; they can explain, justify, reconsider, apologise, offer amends, make changes or take future precautions. Answerability encompasses a richer set of responsibility practices than explainability in computing, or accountability in law.Often, the very act of answering for our actions improves us, helping us be more responsible and trustworthy in the future. This is why answerability is key to bridging responsibility gaps. It is not about who we name as the 'responsible person' (which is more difficult to identify in autonomous systems), but about what we owe to the people holding the system responsible. If the system as a whole (machines + people) can get better at giving the answers that are owed, the system can still meet present and future responsibilities to others. Hence, answerability is a system capability for executing responsibilities that can bridge responsibility gaps. Our ambition is to provide the theoretical and empirical evidence and computational techniques that demonstrate how to enable autonomous systems (including wider "systems" of developers, owners, users, etc) to supply the kinds of answers that people seek from trustworthy agents. Our first workstream establishes the theoretical and conceptual framework that allows answerability to be better understood and executed by system developers, users and regulators. The second workstream grounds this in a people-centred, evidence-driven approach by engaging various publics, users, beneficiaries and regulators of autonomous systems in the research. Focus groups, workshops and interviews will be used to discuss cases and scenarios in health, finance and government that reveal what kinds of answers people expect from trustworthy systems operating in these areas. Finally, our third workstream develops novel computational AI techniques for boosting the answerability of autonomous systems through more dialogical and responsive interfaces with users and regulators. Our research outputs and activities will produce a mix of academic, industry and public-facing resources for designing, deploying and governing more answerable autonomous systems.