Scalable Inference in Emerging Structured Domains

世界上的数据量正在以指数级的速度增长，科学和工业都越来越需要可扩展的工具和技术来分析和理解这些数据。这种需求正在迅速推动机器学习在高科技行业和数据驱动的学科中发挥作用。特别是，机器学习的最新进展给我们带来了深度学习，这是一种改变游戏规则的推理和学习方法，可以高效地处理大量数据，并建立强大和富有表现力的模型。然而，我们不能在要求最高、最令人兴奋的大数据领域使用深度学习。无论我们是考虑大脑的神经连通性，宇宙的大规模结构，还是数据库的关系结构，今天的深度学习都没有工具和技术来解决这些领域中非常高维和结构化的实例。我建议的研究通过设计方法来解决这个关键问题，A)主动对数据进行亚采样以处理非常大的实例，以及B)通过一组不改变其内容的变换来编码领域结构：它的对称性。我将重点放在宇宙学和关系数据这两个高影响领域，以展示在设计深层模型时提出的方法。对于域结构先验未知的设置，我概述了一个计划，通过它的对称性来研究域结构的发现。这一提议的主要成果是简单、通用和有效的方法和模型，这些方法和模型将深度学习的新力量扩展到处理关键结构。如果成功，这一计划将彻底改变目前建立在过于简单化假设基础上的宇宙学方法。我们的目标是实现对关系数据的深度学习，通过提供广泛使用的关系数据库中的可扩展和准确的数据外推技术，将对大数据行业产生重大影响。利用深层模型发现领域不变性的新能力，将为非结构化数据带来可解释性，并将从大量观察中产生浓缩形式的知识。我相信，这个项目为培养真正的跨学科研究人员提供了机会，这不仅有助于保持加拿大在人工智能领域的领先地位，而且还能及时利用这一地位，推动科学和工程领域其他数据驱动的领域的发展。

The amount of data in the world is growing at an exponential rate, and both science and industry increasingly need scalable tools and techniques to analyze and understand it. This need is rapidly promoting the role of machine learning in high-tech industries and data-driven disciplines. In particular, recent advances in machine learning have brought us deep learning, a game-changing approach to inference and learning that can efficiently process a large amount of data and build powerful and expressive models. However, we cannot use deep learning within the most demanding and exciting big-data domains. Whether we are considering the neural connectivity of the brain, the large-scale structure of the Universe, or relational structure of databases, today's deep learning does not have the tools and techniques to address the very high dimensional and structured instances within these domains.My proposed research addresses this critical problem by designing methodologies for A) active subsampling of the data for handling very large instances and; B) encoding domain structure through a group of transformations that do not alter its content: its symmetries. I focus on cosmological and relational data as two high-impact domains to showcase the proposed approach in designing deep models. For the settings in which the domain structure is unknown a priori, I outline a plan to investigate the discovery of domain structure via its symmetries. The major deliverables of this proposal are simple, generic and effective methods and models that extend the newfound power of deep learning to handle key structures. If successful, this program will revolutionize the current approach to cosmology, which is currently built on oversimplifying assumptions. Our objective in enabling deep learning on relational data will have a significant impact on the big-data industry, by providing the technology for scalable and accurate extrapolation of the data in widely used relational databases. The novel ability of discovering domain invariances using deep models, will bring interpretability to unstructured data and will produce a condensed form of knowledge from a large number of observations. I believe this program provides an opportunity for education of truly interdisciplinary researchers that would not only help maintain the leading position of Canada in AI but also leverage this position in a timely fashion to advance other data-driven areas within science and engineering.