RI: Medium: CompCog: Automated Discovery of Macro-Variables from Raw Spatiotemporal Data

RI：中：CompCog：从原始时空数据中自动发现宏变量

• 批准号: 1564330
• 负责人: Pietro Perona
• 金额: $ 110万
• 依托单位: California Institute of Technology
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-05-15 至 2021-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1564330&HistoricalAwards=false
• 关键词: RI; Medium; CompCog; Automated; Discovery

Observation and careful experimentation provide the basis for scientific inquiry, which in turn guides our understanding of the world and policy decisions. Today, scientific data is collected from a vast array of sensors: satellite images and radar, neuro-imaging, microscopes, body monitoring, socio-economic indicators, to name just a few. While models and theories were traditionally derived via careful handcrafting by domain experts, the new data deluge makes direct human analysis impossible. We need intelligent machines that can process vast amounts of sensory data into interpretable quantities that provide actionable information. This project will develop machines that will be able to learn on their own, purely from experience, produce and test hypotheses on causes and effects in complex dynamic scenes, and better collaborate with human scientists and analysts. For generality, we will develop and test our theory in two different domains. Amongst the immediate benefits of our project are methods for discovering the causal relationship between genes, brains and behavior. Our objective is to develop theory and practical algorithms for automatically interpreting a dynamic scene containing interacting agents. This will involve automatically identifying the main spatial locations, the objects, the actors, their actions and goals, and their relations to one another. The output is a description of the events, and hypotheses on the actors? goals, cause-effect relationships and likely developments. The key technical questions that we will tackle are how to infer semantically meaningful "macro" variables (i.e. agents' role and goals, actions, objects, special locations) directly from raw sensory data (mostly video), how to infer the causal relationships among such variables, and how to adaptively plan new experiments, including collecting feedback from human experts, to resolve ambiguities in the model. The intellectual merit of our project lies in developing an end-to-end, pixels-to-causes approach to the automatic analysis of dynamic scenes. To this end, we will integrate, build upon, and transcend the capabilities of extant "low-level" correlational machine learning and "high-level" causal inference approaches, combined with interactive learning approaches to sequential experimental design.

观察和仔细的实验为科学探究提供了基础，而科学探究反过来又指导我们对世界的理解和政策决定。今天，科学数据是从大量的传感器收集的：卫星图像和雷达，神经成像，显微镜，身体监测，社会经济指标，仅举几例。虽然模型和理论传统上是由领域专家精心手工制作的，但新的数据洪流使直接的人类分析变得不可能。我们需要智能机器，可以将大量的感官数据处理成可解释的数量，提供可操作的信息。该项目将开发能够自主学习的机器，纯粹从经验中学习，在复杂的动态场景中产生和测试因果关系的假设，并更好地与人类科学家和分析师合作。一般来说，我们将在两个不同的领域发展和测试我们的理论。我们项目的直接好处之一是发现基因、大脑和行为之间因果关系的方法。我们的目标是开发理论和实用的算法，自动解释一个动态的场景包含交互代理。这将涉及自动识别主要的空间位置、对象、行为者、他们的行动和目标以及他们之间的关系。输出是对事件的描述，以及对演员的假设？目标、因果关系和可能的发展。我们将解决的关键技术问题是如何直接从原始感知数据（主要是视频）推断语义上有意义的“宏观”变量（即代理的角色和目标，动作，对象，特殊位置），如何推断这些变量之间的因果关系，以及如何自适应地计划新的实验，包括收集来自人类专家的反馈，以解决模型中的歧义。 我们项目的智力价值在于开发一种端到端，像素到原因的方法来自动分析动态场景。为此，我们将整合，构建和超越现存的“低级”相关机器学习和“高级”因果推理方法的能力，结合交互式学习方法进行顺序实验设计。