Learning Concepts in Deep Networks

点击翻译按钮获取中文摘要

Learning appropriate representations, or extracting useful featuresfrom data, is one of the fundamental problems of MachineLearning. Recently a number of methods have been developed forlearning deep representations (e.g.\ deep neural networks or deepprobabilistic graphical models). While existing research hasempirically validated the benefit of deep learning, there is currentlya lack in deeper understanding of deep architectures and theirrepresentations. Open questions are: Are deep representationsfundamentally different to kernels, or can they not be understood as aspecial type of kernel? What are characteristics of deeprepresentations that make them beneficial?This project is organized in two parts. In the first (analytical) partwe will develop generative and discriminative methods toanalyze learning concepts in deep networks. We anticipate that thisanalysis will allow for a unified view on kernels and representations,overcoming the false dichotomy between so-called deep and shallowrepresentations. In the second (constructive) part, we will utilizethese analytical tools for developing alternative methods to learndeep representations. Our approach will be to select good deeprepresentations from massive sets of randomized proposal structuresusing the analytical measures to be developed initially.The outcome of this project will be (1) methods to precisely quantifythe characteristics and benefits of deep learning and (2)concepts for constructing improved deep learning based on thesemeasures.