Collaborative Research: Loopholes as a window into the learning of meaning

智能机器可以帮助实现人类的主要目标。但即使是目前最先进的机器也可能会灾难性地误解它们被要求做的事情，导致机器“做你要求的事情，但不是你想要的”。与这些人机交互的失败相反，从很小的时候起，人类就可以快速有效地传达他们的目标，并找到合作和帮助的方法。但是当人们的价值观不一致时，他们就会找到“漏洞”来避免合作或遵守。漏洞提供了一个独特的窗口，可以看到成功但不透明的目标理解常识过程。虽然漏洞是一个普遍存在的日常关注与真实的世界的影响，有很少的计算或认知研究检查这一现象。这个项目的目的是研究心理过程，使人类直观地和有目的地扭曲通信的漏洞行为。这项研究将有助于解决安全智能机器和人机交互设计中的核心开放挑战，并将提高我们对社交互动出现的理解。 以前的研究主要集中在儿童如何学习社交和协商价值观，但没有关注儿童和成人如何处理和利用价值观错位。这给认知科学和人机交互提出了一个关键问题：人们如何学会从模糊的沟通到预期目标、合理的替代方案和自己的价值观的一致性？对发展的研究在回答这一问题时特别重要，因为发展轨迹揭示了哪些过程是这种能力的基础，哪些过程是随着更多的知识和经验逐渐形成的。该项目结合了人工智能，计算认知科学和社会认知发展的方法，并将（1）使用公民科学和公共数据的大型开放数据库来描述漏洞的出现和范围，（2）建立一个正式的框架，由数据来建模解释和（错误）从稀疏语句中对齐社会目标，（3）通过不同人群的对照实验验证框架，以研究从童年到成年的漏洞寻找评估，以及（4）通过对人类机器推断目标的新研究来扩展这一框架-该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Intelligent machines could help achieve major human goals. But even current state-of-the-art machines can catastrophically misunderstand what they were asked to do, resulting in machines that 'do what you asked, but not what you want'. In contrast to these failures of human-machine interactions, from an early age humans can quickly and efficiently communicate their goals, and find ways to cooperate and help. But when people's values do not align, they find ‘loopholes’ to avoid cooperating or complying. Loopholes offer a unique window into the successful but opaque commonsense process of goal understanding. While loopholes are a pervasive everyday concern with real world implications, there is little computational or cognitive research examining this phenomenon. This project means to study the mental processes that allow humans to intuitively and purposefully contort communication in loophole-behavior. This research will help tackle central open challenges in the design of safe intelligent machines and human-technology interactions, and will improve our understanding of the emergence of social interactions. Previous research has focused on how children learn to communicate socially and negotiate values, but not on how children and adults handle and exploit value misalignment. This raises a crucial question for cognitive science and human-machine interactions: how do people learn to go from ambiguous communication to the alignment of intended goals, plausible alternatives, and one’s own values? Studies of development are particularly important in answering this question, as the developmental trajectory sheds light on which processes are foundational to this ability and which are brought in piecemeal with greater knowledge and experience. The project combines methods from AI, computational cognitive science, and social cognitive development, and will (1) characterize the emergence and scope of loopholes in the wild with large open databases using citizen-science and public data, (2) build a formal framework informed by the data for modeling the interpretation and (mis)alignment of social goals from sparse statements, (3) validate the framework using controlled experiments with diverse populations to study the evaluation of loophole-seeking from childhood to adulthood, and (4) extend this framework with novel studies on the inferred goals of machines in human-machine interactions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.