Learning Concepts in Deep Networks

学习适当的表示，或从数据中提取有用的特征，是机器学习的基本问题之一。最近已经开发了许多方法来学习深度表示（例如，深度神经网络或深度概率图形模型）。虽然现有的研究已经从经验上验证了深度学习的好处，但目前缺乏对深度架构及其表示的更深入理解。 开放的问题是：深层表示与内核有本质的不同吗？或者它们不能被理解为一种特殊类型的内核？ 什么样的特征使它们有益？该项目分为两部分。在第一部分（分析）中，我们将开发生成和判别方法来分析深度网络中的学习概念。我们预计，这种分析将允许一个统一的看法内核和表示，克服所谓的深和浅表示之间的错误二分法。 在第二（建设性）部分，我们将利用这些分析工具来开发替代方法来学习深度表示。我们的方法将是使用最初开发的分析方法从大量随机提案结构中选择好的深度呈现。这个项目的成果将是（1）精确量化深度学习的特征和好处的方法，以及（2）基于这些方法构建改进的深度学习的概念。

Learning appropriate representations, or extracting useful featuresfrom data, is one of the fundamental problems of MachineLearning. Recently a number of methods have been developed forlearning deep representations (e.g.\ deep neural networks or deepprobabilistic graphical models). While existing research hasempirically validated the benefit of deep learning, there is currentlya lack in deeper understanding of deep architectures and theirrepresentations. Open questions are: Are deep representationsfundamentally different to kernels, or can they not be understood as aspecial type of kernel? What are characteristics of deeprepresentations that make them beneficial?This project is organized in two parts. In the first (analytical) partwe will develop generative and discriminative methods toanalyze learning concepts in deep networks. We anticipate that thisanalysis will allow for a unified view on kernels and representations,overcoming the false dichotomy between so-called deep and shallowrepresentations. In the second (constructive) part, we will utilizethese analytical tools for developing alternative methods to learndeep representations. Our approach will be to select good deeprepresentations from massive sets of randomized proposal structuresusing the analytical measures to be developed initially.The outcome of this project will be (1) methods to precisely quantifythe characteristics and benefits of deep learning and (2)concepts for constructing improved deep learning based on thesemeasures.