Collaborative Research: CompCog: Adversarial Collaborative Research on Intuitive Physical Reasoning

人们能够以令人惊讶的复杂方式对世界进行推理，但我们认为这些能力是简单的“常识”的一部分，通常在个人和文化中都是如此。我们扔接球，把盘子堆在水槽里，几乎毫不费力地倒上一杯早晨的咖啡。然而，支持这些能力的认知系统并没有得到很好的理解；即使是我们在机器人中对它们进行反向工程的最先进尝试，也达不到人类水平的效率或灵活性。这笔资金被设计为一种“对抗性合作”，旨在将来自两个不同方面的科学家聚集在一起，就人类物理推理能力的性质展开关键辩论。一种理论(由麻省理工学院PI支持)认为，这种物理推理基于一种认知系统，允许人们模拟接下来可能发生的事情，类似于视频游戏的物理引擎被用来预测这些场景中接下来会发生什么。虽然这一理论已经为人类行为提供了许多成功的解释，包括准确预测人们认为Jenga塔楼将如何倒塌，或者他们认为球将落在空中的哪里，但另一个不断增长的研究机构(由纽约大学PI领导)已经证明了许多例子，在这些例子中，模拟理论不能充分描述人的行为，但更简单和近似的经验法则(即使是不准确的)可以。由于人类的物理推理不太可能是纯粹的模拟或纯粹基于简化的规则，来自这场辩论双方的专家团队将对促进我们对构成这些推理能力的认知过程的理解至关重要。为了调和这些观点，这项拨款提出了这样一种想法，即考虑已知的人类局限性--例如，在记忆或注意力上--可以解释人们在对物理世界进行推理时所使用的过程。我们的目标是将这些限制整合到一个更完整的人类推理理论中，该理论既可以解释我们在理解物理世界方面的失败，也可以解释我们的成功。对这些过程的真正理解将需要通过设计具有与人类相似的限制和能力的计算模型来对人类的认知和感知进行“逆向工程”。这些科学模型可能会为人工智能和机器人领域的研究人员提供洞察力，这些研究人员对设计像人类一样与世界互动的系统感兴趣，包括自动驾驶汽车或假肢控制。此外，探索人们是如何学习和推理物理的，可以为物理教育提供新的途径。最后，研究和建模物理推理的这些方面将需要开发可扩展的工具，这些工具将作为开源软件发布，以向更多的科学家开放对人类物理推理的研究。该项目研究并提出解决人类物理推理理论之间的紧张关系，这些理论认为它基于相对准确的可模拟心理模型，而那些理论认为它基于启发式和其他定性推理形式。这项研究包括与那些用于演示模拟理论的实验相关的实验，但这些实验经过修改，以两种截然不同的方式诱导物理推理的捷径。目标1实验考虑了预计会遇到人力资源限制的场景，无论是在注意力、记忆力还是时间上-例如，要求人们预测复杂的积木塔的稳定性，这些积木塔有太多的碎片无法单独跟踪。Aim 2实验考虑了一些场景，这些场景可以通过模拟推理，但更容易用简单的规则或启发式推理--例如，研究人们在判断堆满物体的平衡木将朝哪个方向下落时，如何使用像“较重的一侧会倾倒”这样的规则。在这些实验中，根据预期的资源限制(例如，使用规则、聚焦于对象的子集或更粗略地表示对象)来检查人类行为是否偏离纯模拟理论，并且开发计算模型来解释该行为。这些模型是围绕“资源理性”认知的框架设计的，该框架建议人们以一种有效地解决他们所遇到的问题的方式来配置有限的认知资源。行为结果和模型一起可以调查(A)人们的身体推理是否以及何时受到资源限制的限制，以及(B)人们绕过这些限制所采取的捷径类型。进行这项研究需要开发一个集成的软件套件，用于在各种物理场景中设计实验和建模。设计这些集成包通常需要大量的技术--物理模拟器、图形引擎、计算建模方法--这些技术超出了大多数心理学家的能力范围，这反过来又限制了对人类物理推理的研究。私人投资机构在这方面的贡献是独一无二的，因为他们的实验室专注于心理学的计算模型，而且他们在开发开放源码软件方面有着广泛的记录，这些软件被世界各地的多个研究小组使用。所使用的软件套件是开源的，并与更广泛的研究社区共享，以促进对人类物理推理的进一步研究，而不需要对底层技术的广泛知识。这项工作得到了SBE/BCS感知、行动和认知、EHR核心研究(ECR)和CSE/IIS稳健智能的支持。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

People are able to reason about the world in amazingly complex ways, yet we consider these capacities part of simple “common sense,” generally shared across individuals and cultures. We toss and catch balls, stack dishes in the sink, and pour a morning cup of coffee with almost no effort. Yet the cognitive systems that support these capabilities are not well understood; even our most advanced attempts to reverse engineer them in robots fall short of human-level efficiency or flexibility. This grant was designed as an “adversarial collaboration” to bring together scientists from two different sides of a critical debate about the nature of human physical reasoning abilities. One theory (championed by the MIT PIs) suggests that this physical reasoning is based on a cognitive system that allows people to simulate what might happen next, similar to how physics engines for video games are used to predict what will happen next in those scenes. While this theory has provided many successful explanations of human behavior, including making precise predictions about how people think Jenga towers will fall, or where they think balls flying through the air will land, another growing body of research (led by the NYU PIs) has demonstrated many instances where the simulation theory cannot adequately describe what people do, but where simpler and approximate “rules-of-thumb” (even inaccurate ones) can. Because human physical reasoning is unlikely to be purely simulation or purely based on simplified rules, a team of experts from both sides of this debate will be crucial for advancing our understanding of the cognitive processes that underlie these reasoning capabilities. Towards reconciling these views, this grant advances the idea that consideration of known human limitations -- e.g., in memory or attention -- can explain the processes that people use when reasoning about the physical world. The goal is to integrate these constraints into a more complete theory of human reasoning that can account for both our failures and our successes in comprehending the physical world. True understanding of these processes will require “reverse engineering” human cognition and perception by designing computational models with similar limitations and capabilities to people. These scientific models may provide insight for researchers in AI and robotics who are interested in designing systems that interact with the world like people, including self-driving cars or the control of prosthetic limbs. Furthermore, exploring how people learn and reason about physics may provide new approaches for physics education. Finally, studying and modeling these facets of physical reasoning will require developing extensible tools, which will be released as open-source software to open up the research into human physical reasoning to a wider set of scientists.This project studies and proposes to resolve tensions between theories of human physical reasoning that suggest that it is based on relatively accurate simulatable mental models, and those that suggest it is based on heuristics and other qualitative forms of reasoning. The research includes experiments related to those that have been used to demonstrate simulation theory, but modified to induce shortcuts in physical reasoning in two broadly different ways. Aim 1 experiments consider scenarios that are expected to run into human resource limitations, either in attention, memory, or time – for instance, asking people to predict the stability of complex towers of blocks with too many pieces to track individually. Aim 2 experiments consider scenarios that could be reasoned about with simulation, but could more easily be reasoned about with simple rules or heuristics – for instance, studying how people use rules like “the heavier side will tip over” when judging which direction a balance beam stacked with objects will fall. Human behavior in these experiments is examined for deviations from pure simulation theory in line with the expected resource limitations (e.g., using rules, focusing on a subset of objects, or representing objects more coarsely), and computational models are developed to explain this behavior. These models are designed around the framework of “resource-rational” cognition, which suggests that people deploy limited cognitive resources in a way that efficiently solves the problems they encounter. The behavioral results and models together allow investigation into (a) whether and when people’s physical reasoning is constrained by resource limitations, and (b) the types of shortcuts people take to circumvent these limitations. Performing this research requires developing an integrated software suite for designing experiments and modeling across a wide variety of physical scenarios. Designing these integrated packages typically requires a large set of technologies -- physics simulators, graphics engines, computational modeling methods -- that are outside the reach of most psychologists, which in turn limits research into human physical reasoning. The PIs are in a unique position to contribute here because their laboratories are focused on computational models of psychology and they have an extensive track record of developing open-source software used by multiple research groups worldwide. The software suite used is designed to be open-sourced and shared with the broader research community to facilitate further research into human physical reasoning without requiring extensive knowledge of the underlying technologies.This work was supported by SBE/BCS Perception, Action, and Cognition, EHR Core Research (ECR), and CISE/IIS Robust Intelligence.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.