Collaborative Research: Loopholes as a window into the learning of meaning

智能机器可以帮助实现人类的主要目标。但是，即使是目前最先进的机器也会对它们被要求做的事情产生灾难性的误解，导致机器“做你要求的，而不是你想要的”。与这些人机交互的失败相比，人类从很小的时候就可以快速有效地沟通他们的目标，并找到合作和帮助的方法。但当人们的价值观不一致时，他们就会找到“漏洞”来避免合作或遵守。漏洞为了解成功但不透明的目标理解常识过程提供了一个独特的窗口。虽然漏洞是现实世界中普遍存在的问题，但很少有计算或认知研究来研究这一现象。这个项目的目的是研究允许人类在漏洞行为中直观地、有目的地扭曲交流的心理过程。这项研究将有助于解决安全智能机器和人机交互设计中的核心开放挑战，并将提高我们对社会互动出现的理解。之前的研究关注的是儿童如何学会社交沟通和价值观协商，而不是儿童和成人如何处理和利用价值观偏差。这为认知科学和人机交互提出了一个至关重要的问题：人们如何学会从模棱两可的沟通转向预期目标、合理的替代方案和自己的价值观？对发展的研究对于回答这个问题特别重要，因为发展轨迹阐明了哪些过程是这种能力的基础，哪些是随着更多的知识和经验而逐渐引入的。该项目结合了人工智能、计算认知科学和社会认知发展的方法，并将(1)使用使用公民科学和公共数据的大型开放数据库来描述野外漏洞的出现和范围，(2)建立一个由数据提供信息的正式框架，用于从稀疏陈述中对社会目标的解释和（错误）一致性建模。(3)利用不同人群的对照实验来验证该框架，研究从童年到成年的漏洞寻找评估；(4)通过对人机交互中机器推断目标的新研究来扩展该框架。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Intelligent machines could help achieve major human goals. But even current state-of-the-art machines can catastrophically misunderstand what they were asked to do, resulting in machines that 'do what you asked, but not what you want'. In contrast to these failures of human-machine interactions, from an early age humans can quickly and efficiently communicate their goals, and find ways to cooperate and help. But when people's values do not align, they find ‘loopholes’ to avoid cooperating or complying. Loopholes offer a unique window into the successful but opaque commonsense process of goal understanding. While loopholes are a pervasive everyday concern with real world implications, there is little computational or cognitive research examining this phenomenon. This project means to study the mental processes that allow humans to intuitively and purposefully contort communication in loophole-behavior. This research will help tackle central open challenges in the design of safe intelligent machines and human-technology interactions, and will improve our understanding of the emergence of social interactions. Previous research has focused on how children learn to communicate socially and negotiate values, but not on how children and adults handle and exploit value misalignment. This raises a crucial question for cognitive science and human-machine interactions: how do people learn to go from ambiguous communication to the alignment of intended goals, plausible alternatives, and one’s own values? Studies of development are particularly important in answering this question, as the developmental trajectory sheds light on which processes are foundational to this ability and which are brought in piecemeal with greater knowledge and experience. The project combines methods from AI, computational cognitive science, and social cognitive development, and will (1) characterize the emergence and scope of loopholes in the wild with large open databases using citizen-science and public data, (2) build a formal framework informed by the data for modeling the interpretation and (mis)alignment of social goals from sparse statements, (3) validate the framework using controlled experiments with diverse populations to study the evaluation of loophole-seeking from childhood to adulthood, and (4) extend this framework with novel studies on the inferred goals of machines in human-machine interactions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.