RI: Medium: CompCog: Automated Discovery of Macro-Variables from Raw Spatiotemporal Data

观察和仔细的实验为科学探究提供了基础，而科学探究反过来又指导我们对世界和政策决策的理解。今天，科学数据是从大量传感器收集的：卫星图像和雷达、神经成像、显微镜、身体监测、社会经济指标，仅举几例。虽然模型和理论传统上是由领域专家精心制作得出的，但新的数据洪流使直接的人工分析变得不可能。我们需要能够将大量感官数据处理成可解释数量并提供可操作信息的智能机器。该项目将开发能够完全从经验中自主学习的机器，在复杂的动态场景中产生和测试因果假设，并更好地与人类科学家和分析师合作。为了通用性，我们将在两个不同的领域发展和测试我们的理论。我们项目的直接好处之一是发现基因、大脑和行为之间因果关系的方法。我们的目标是开发理论和实用算法来自动解释包含交互代理的动态场景。这将涉及自动识别主要空间位置、对象、参与者、他们的行动和目标，以及他们彼此之间的关系。输出是对事件的描述，以及对参与者的假设。目标，因果关系和可能的发展。我们将解决的关键技术问题是如何直接从原始感官数据（主要是视频）中推断语义上有意义的“宏观”变量（即代理的角色和目标，动作，对象，特殊位置），如何推断这些变量之间的因果关系，以及如何自适应地计划新的实验，包括收集来自人类专家的反馈，以解决模型中的歧义。我们项目的智力价值在于开发一种端到端、像素到原因的方法来自动分析动态场景。为此，我们将整合、建立并超越现有的“低级”相关机器学习和“高级”因果推理方法的能力，并结合交互式学习方法进行顺序实验设计。

Observation and careful experimentation provide the basis for scientific inquiry, which in turn guides our understanding of the world and policy decisions. Today, scientific data is collected from a vast array of sensors: satellite images and radar, neuro-imaging, microscopes, body monitoring, socio-economic indicators, to name just a few. While models and theories were traditionally derived via careful handcrafting by domain experts, the new data deluge makes direct human analysis impossible. We need intelligent machines that can process vast amounts of sensory data into interpretable quantities that provide actionable information. This project will develop machines that will be able to learn on their own, purely from experience, produce and test hypotheses on causes and effects in complex dynamic scenes, and better collaborate with human scientists and analysts. For generality, we will develop and test our theory in two different domains. Amongst the immediate benefits of our project are methods for discovering the causal relationship between genes, brains and behavior. Our objective is to develop theory and practical algorithms for automatically interpreting a dynamic scene containing interacting agents. This will involve automatically identifying the main spatial locations, the objects, the actors, their actions and goals, and their relations to one another. The output is a description of the events, and hypotheses on the actors? goals, cause-effect relationships and likely developments. The key technical questions that we will tackle are how to infer semantically meaningful "macro" variables (i.e. agents' role and goals, actions, objects, special locations) directly from raw sensory data (mostly video), how to infer the causal relationships among such variables, and how to adaptively plan new experiments, including collecting feedback from human experts, to resolve ambiguities in the model. The intellectual merit of our project lies in developing an end-to-end, pixels-to-causes approach to the automatic analysis of dynamic scenes. To this end, we will integrate, build upon, and transcend the capabilities of extant "low-level" correlational machine learning and "high-level" causal inference approaches, combined with interactive learning approaches to sequential experimental design.