Collaborative Research: Loopholes as a window into the learning of meaning

合作研究：漏洞作为意义学习的窗口

• 批准号: 2118096
• 负责人: Tomer Ullman
• 金额: $ 37.22万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118096&HistoricalAwards=false
• 关键词: Collaborative; Research; Loopholes; window; into

Intelligent machines could help achieve major human goals. But even current state-of-the-art machines can catastrophically misunderstand what they were asked to do, resulting in machines that 'do what you asked, but not what you want'. In contrast to these failures of human-machine interactions, from an early age humans can quickly and efficiently communicate their goals, and find ways to cooperate and help. But when people's values do not align, they find ‘loopholes’ to avoid cooperating or complying. Loopholes offer a unique window into the successful but opaque commonsense process of goal understanding. While loopholes are a pervasive everyday concern with real world implications, there is little computational or cognitive research examining this phenomenon. This project means to study the mental processes that allow humans to intuitively and purposefully contort communication in loophole-behavior. This research will help tackle central open challenges in the design of safe intelligent machines and human-technology interactions, and will improve our understanding of the emergence of social interactions. Previous research has focused on how children learn to communicate socially and negotiate values, but not on how children and adults handle and exploit value misalignment. This raises a crucial question for cognitive science and human-machine interactions: how do people learn to go from ambiguous communication to the alignment of intended goals, plausible alternatives, and one’s own values? Studies of development are particularly important in answering this question, as the developmental trajectory sheds light on which processes are foundational to this ability and which are brought in piecemeal with greater knowledge and experience. The project combines methods from AI, computational cognitive science, and social cognitive development, and will (1) characterize the emergence and scope of loopholes in the wild with large open databases using citizen-science and public data, (2) build a formal framework informed by the data for modeling the interpretation and (mis)alignment of social goals from sparse statements, (3) validate the framework using controlled experiments with diverse populations to study the evaluation of loophole-seeking from childhood to adulthood, and (4) extend this framework with novel studies on the inferred goals of machines in human-machine interactions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

智能机器可以帮助实现人类的主要目标。但即使是目前最先进的机器也可能灾难性地误解它们被要求做的事情，导致机器“做你要求的事情，而不是你想要的事情”。与这些人机交互的失败相反，人类从小就可以快速有效地交流他们的目标，并找到合作和帮助的方法。但当人们的价值观不一致时，他们就会找到“漏洞”，以避免合作或顺从。漏洞为理解目标的成功但不透明的常识过程提供了一个独特的窗口。虽然漏洞是日常生活中普遍存在的问题，具有现实世界的影响，但很少有计算或认知研究来检验这一现象。这个项目意味着研究允许人类在漏洞行为中直观和有目的地扭曲交流的心理过程。这项研究将有助于解决安全智能机器和人-技术互动设计中的中心开放挑战，并将提高我们对社交互动出现的理解。此前的研究侧重于儿童如何学会社交和协商价值观，而不是儿童和成年人如何处理和利用价值观失调。这对认知科学和人机交互提出了一个至关重要的问题：人们如何学会从模棱两可的交流转向预期目标、合理的替代方案和自己的价值观的一致？对发展的研究对于回答这一问题特别重要，因为发展轨迹揭示了哪些过程是这种能力的基础，哪些过程是随着更多的知识和经验零碎地引入的。该项目结合了人工智能、计算认知科学和社会认知发展的方法，将(1)利用公民科学和公共数据，利用大型开放数据库表征野外漏洞的出现和范围，(2)建立一个由数据提供信息的正式框架，以模拟从稀疏语句到社会目标的解释和(误)对齐，(3)使用不同人群的受控实验来验证该框架，以研究从儿童到成人对寻找漏洞的评价，以及(4)扩展这一框架，对人机交互中机器的推断目标进行新的研究。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Skirting the Sacred: Moral Violations Make Intentional Misunderstandings Worse

回避神圣：道德违规使故意的误解变得更糟

• 发表时间: 2023
• 期刊: Proceedings of the Annual Meeting of the Cognitive Science Society
• 作者: Parece, K.

Loopholes, a Window into Value Alignment and the Learning of Meaning

漏洞、价值调整和意义学习的窗口

• 发表时间: 2023
• 期刊: Proceedings of the Annual Meeting of the Cognitive Science Society,
• 作者: Bridgers, S.

Comparing the Evaluation and Production of Loophole Behavior in Children and Large Language Models.

比较儿童和大型语言模型中漏洞行为的评估和产生。

• 发表时间: 2023
• 期刊: ICML
• 作者: Murthy, S. K.