CHS: Small: Collaborative Research: Role-Based Norm Violation Response in Human-Robot Teams

CHS：小型：协作研究：人机团队中基于角色的规范违规响应

• 批准号: 1909847
• 负责人: Thomas Williams
• 金额: $ 50万
• 依托单位: Colorado School of Mines
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909847&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; Role

Robots may need to carefully decide when and how to reject commands given to them, if the actions required to carry out those commands are not morally permissible. Most previous work on this topic takes a norm-based ethical approach, where a robot would operate under a set of rules describing what states or actions are morally wrong, and use those rules to explain its actions. In contrast, this project explores a role-based perspective, in which the robot reasons about the relationships it holds with others, the roles it plays in those relationships, and whether the actions requested of it are benevolent with respect to those roles and relationships. Specifically, the researchers will develop a framework to allow robots to reason in this way and generate explanations of its actions based on this reasoning. The researchers will then explore how role-based and norm-based command rejections compare in terms of how they affect human-robot teamwork, and design algorithms to allow robots to automatically decide what type of rejection to generate based on their context. These algorithms and explanations will be evaluated in two very different contexts with different types of relationships, roles, and rules: with civilian undergraduates at the Colorado School of Mines, and with Air Force cadets at the US Air Force Academy. This work will not only increase robots' ability to behave ethically and act as good teammates, but will also advance moral philosophy by providing experimental evidence for the relative importance and effectiveness of different tenets of role-based moral philosophy.More formally, the goals of this research are to investigate context-sensitive tradeoffs between rule-based and role-based responses, and the representations and mechanisms needed to facilitate role-based responses. The research team will do this by identifying metrics to assess response acceptability, quality, and effectiveness; modeling the generation of role-based responses and selection between role-based and rule-based responses; conducting experiments to validate those models and responses; and using the results to articulate novel moral and philosophical arguments. The team will start with exploratory studies at each experimental site contrasting the effectiveness of different command rejection phrasings formed by crossing different Speech-Act Theoretic communication strategies paired with different moral philosophical backgrounds, with respect to (a) field-standard survey measures of trust, likability, mindfulness, workload, and norm strength; (b) qualitative analysis of video data; and (c) statistical linguistic analyses from the multimodal interaction community. The researchers will then develop a framework for robots to generate these responses that will involve formal computational model development of inter-team relations, roles, and actions; machine learning-based modeling of norm violation response strategy selection using features such as task context characterization, role-theoretic proposed action benevolence, and expected responses to the responses; and integration into the DIARC robot cognitive architecture. Based on this work, the team will both advance traditional moral philosophical arguments and develop a novel framework in which moral philosophical arguments are justified based on the effects of computational models on moral psychology. Overall, the project will lead to foundational, interdisciplinary knowledge into norm violation response, consisting of algorithms for selecting between norm violation responses grounded in different ethical theories, guidelines for and insights into the design of morally competent language capable robots, novel computational accounts of role-based robot ethics, and novel empirically informed moral philosophical arguments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

如果执行这些命令所需的行动在道德上是不允许的，机器人可能需要仔细决定何时以及如何拒绝向它们发出的命令。以前关于这一主题的大多数工作都采取了基于规范的伦理方法，即机器人将在一套描述哪些状态或行为在道德上是错误的规则下操作，并使用这些规则来解释它的行为。相比之下，这个项目探索了基于角色的观点，在这种观点中，机器人推理它与他人之间的关系，它在这些关系中扮演的角色，以及要求它的行动是否相对于这些角色和关系是仁慈的。具体地说，研究人员将开发一个框架，允许机器人以这种方式进行推理，并根据这种推理产生对其行为的解释。然后，研究人员将探索基于角色的命令拒绝和基于规范的命令拒绝在如何影响人与机器人的团队合作方面的比较，并设计算法，允许机器人根据其上下文自动决定生成哪种类型的拒绝。这些算法和解释将在两个非常不同的背景下进行评估，它们具有不同类型的关系、角色和规则：与科罗拉多矿业学院的平民本科生和美国空军学院的空军学员。这项工作不仅将提高机器人的道德行为能力和良好的队友行为，还将通过提供实验证据，证明基于角色的道德哲学的不同信条的相对重要性和有效性，从而促进道德哲学的发展。更正式地，本研究的目标是调查基于规则的反应和基于角色的反应之间的上下文敏感权衡，以及促进基于角色的反应所需的表征和机制。研究团队将通过确定评估响应的可接受性、质量和有效性的指标来实现这一点；对基于角色的响应的生成和基于角色的响应和基于规则的响应之间的选择进行建模；进行实验以验证这些模型和响应；并使用结果来阐明新的道德和哲学论点。该团队将在每个试验点进行探索性研究，对比不同言语行为理论沟通策略与不同道德哲学背景形成的不同命令拒绝措辞的有效性，涉及以下方面：(A)现场标准调查测量信任、可爱性、专注力、工作量和规范强度；(B)视频数据的定性分析；以及(C)来自多模式互动社区的统计语言分析。然后，研究人员将为机器人开发一个框架来生成这些响应，其中将包括团队间关系、角色和动作的正式计算模型开发；使用任务上下文表征、角色理论建议的动作仁慈和对响应的预期响应等特征对违反规范的响应策略选择进行基于机器学习的建模；以及集成到Diarc型机器人认知体系结构中。在这项工作的基础上，该团队将既推进传统的道德哲学论点，又开发一种新的框架，其中道德哲学论点是基于计算模型对道德心理学的影响而被证明是合理的。总体而言，该项目将把基础的、跨学科的知识转化为规范违反反应，包括在基于不同伦理理论的规范违反反应之间进行选择的算法、具有道德能力的语言能力机器人的设计指南和见解、基于角色的机器人伦理的新颖计算账户以及新颖的经验性道德哲学论证。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

The Rule of Virtue: A Confucian Response to the Ethical Challenges of Technocracy

• DOI: 10.1007/s11948-021-00341-6
• 发表时间: 2021-10-01
• 期刊: SCIENCE AND ENGINEERING ETHICS
• 影响因子: 3.7
• 作者: Lan, Lishan;Zhu, Qin;Liu, Yongmou
• 通讯作者: Liu, Yongmou

Comparing Norm-Based and Role-Based Strategies for Robot Communication of Role-Grounded Moral Norms

比较基于规范和基于角色的机器人通信基于角色的道德规范策略

• DOI: 10.1145/3571719
• 发表时间: 2023
• 期刊: ACM Transactions on Human-Robot Interaction
• 影响因子: 5.1
• 作者: Wen, Ruchen;Kim, Boyoung;Phillips, Elizabeth;Zhu, Qin;Williams, Tom
• 通讯作者: Williams, Tom

Why and How Robots Should Say ‘No’

• DOI: 10.1007/s12369-021-00780-y
• 发表时间: 2021-05
• 期刊: International Journal of Social Robotics
• 影响因子: 4.7
• 作者: Gordon Briggs;T. Williams;R. Jackson;Matthias Scheutz

Unpretty Please: Ostensibly Polite Wakewords Discourage Politeness in both Robot-Directed and Human-Directed Communication

不漂亮的请：表面上礼貌的唤醒词不鼓励机器人主导和人类主导的沟通中的礼貌

• DOI: 10.1145/3536221.3556615
• 发表时间: 2022
• 期刊: International Conference on Multimodal Interaction
• 作者: Wen, Ruchen;Barton, Brandon;Fauré, Sebastian;Williams, Tom
• 通讯作者: Williams, Tom

Toward Hybrid Relational-Normative Models of Robot Cognition

走向机器人认知的混合关系规范模型

• DOI: 10.1145/3434074.3446353
• 发表时间: 2021
• 期刊: ACM/IEEE International Conference on Human-Robot Interaction
• 作者: Wen, Ruchen