RI: Small: Unraveling and Building Top-Down Generators in Deep Convolutional Neural Networks

深度学习最近显著推进了与人工智能密切相关的研究领域。然而，知识表示的基本问题仍然是开放的，自上而下的过程在深度学习中的作用还不是很清楚。例如，为了训练深度学习算法来检测图像中狗的平移，训练深度学习的数据驱动方式需要通过在图像中移动狗来生成数千个样本。然而，自上而下的模型（如果可用）可以使用沿轴的两个变量沿着直接检测平移。该项目的主要目标是探索一条发现、学习和构建嵌入式深度学习模型的路径，以解释卷积神经网络中丰富的自上而下空间变换和几何组合。由此产生的模型提供了一种通过神经网络层理解嵌入式自顶向下转换过程的透明方式。学习神经启发的自上而下的知识表示将有利于多个学科的研究，包括视觉感知，脑科学，认知建模和决策。目前的深度学习实践，例如卷积神经网络（CNN），主要由数据驱动的自下而上方法主导。虽然使用卷积神经网络（CNN）的各种应用的性能令人印象深刻，但在底层CNN所能提供的与全面智能所需的之间存在着很大的差距。这些强烈的自下而上的CNN特征为深度学习提供了一个很大的空间，使其能够整合自上而下的信息，以实现有效的知识表示、网络学习、认知建模和视觉推理。这个项目是关于建立一个开发自顶向下生成器的路线图。这是通过揭示显式自上而下的知识表示和传播的作用，通过研究卷积神经网络内部产生的特征流，通过构建联合收割机自上而下和自下而上过程的强大的综合分析方法，以及通过创建显式生成模型来帮助广泛的应用。研究自上而下的生成器对广泛的应用程序家族的好处非常有趣，包括但不限于：创建网络内部数据增强、构建对象检测、开发场景理解系统;对组合和上下文对象配置进行建模;以及执行零射击学习。

Deep learning has recently significantly advanced research fields that are closely related to artificial intelligence. The fundamental problem of knowledge representation however remains open and the role of top-down process in deep learning is yet not very clear. For example, to train a deep learning algorithm to detect simply the translation of a dog in an image, a data-driven way of training deep learning would require generating thousands of samples by moving the dog around in the image. However, a top-down model, if available, can directly detect translation using two variables along the axes. The main goal of this project is to explore a path to discover, learn, and build embedded deep learning models, accounting for a rich family of top-down spatial transformation and geometric composition in convolutional neural networks. The resulting models provide a transparent way of understanding the embedded top-down transformation process through neural network layers. The learned neurally-inspired top-down knowledge representation will benefit studies across multiple disciplines, including visual perception, brain sciences, cognitive modeling, and decision making. The current practice in deep learning, for example convolutional neural networks (CNN), is largely dominated by data-driven bottom-up approaches. While the performances of various applications using convolutional neural networks (CNN) are impressive, there nevertheless exists a big gap between what bottom CNN can offer and what comprehensive intelligence requires. These strongly bottom-up CNN characteristics leave a big room for one to provide deep learning with the ability to also incorporate top-down information for effective knowledge representation, network learning, cognitive modeling, and visual inference. This project is about building a roadmap towards developing top-down generators. This is done by unraveling the role of explicit top-down knowledge representation and propagation, by studying the feature flows produced inside the convolutional neural networks, by building robust analysis-by-synthesis methods that combine top-down and bottom-up processes, and by creating explicit generative models to assist a wide range of applications. The benefit of studying the top-down generators to a broad family of applications is greatly intriguing, including but not limited to: creating network internal data augmentation, building object detection, developing scene understanding systems; modeling compositional and contextual object configurations; and performing zero-shot learning.