RI: Medium: CompCog: Automated Discovery of Macro-Variables from Raw Spatiotemporal Data

点击翻译按钮获取中文摘要

Observation and careful experimentation provide the basis for scientific inquiry, which in turn guides our understanding of the world and policy decisions. Today, scientific data is collected from a vast array of sensors: satellite images and radar, neuro-imaging, microscopes, body monitoring, socio-economic indicators, to name just a few. While models and theories were traditionally derived via careful handcrafting by domain experts, the new data deluge makes direct human analysis impossible. We need intelligent machines that can process vast amounts of sensory data into interpretable quantities that provide actionable information. This project will develop machines that will be able to learn on their own, purely from experience, produce and test hypotheses on causes and effects in complex dynamic scenes, and better collaborate with human scientists and analysts. For generality, we will develop and test our theory in two different domains. Amongst the immediate benefits of our project are methods for discovering the causal relationship between genes, brains and behavior. Our objective is to develop theory and practical algorithms for automatically interpreting a dynamic scene containing interacting agents. This will involve automatically identifying the main spatial locations, the objects, the actors, their actions and goals, and their relations to one another. The output is a description of the events, and hypotheses on the actors? goals, cause-effect relationships and likely developments. The key technical questions that we will tackle are how to infer semantically meaningful "macro" variables (i.e. agents' role and goals, actions, objects, special locations) directly from raw sensory data (mostly video), how to infer the causal relationships among such variables, and how to adaptively plan new experiments, including collecting feedback from human experts, to resolve ambiguities in the model. The intellectual merit of our project lies in developing an end-to-end, pixels-to-causes approach to the automatic analysis of dynamic scenes. To this end, we will integrate, build upon, and transcend the capabilities of extant "low-level" correlational machine learning and "high-level" causal inference approaches, combined with interactive learning approaches to sequential experimental design.