A tutorial survey of architectures, algorithms, and applications for deep learning
A tutorial survey of architectures, algorithms, and applications for deep learning
复制标题
DOI：
10.1017/atsip.2013.9
复制
发表时间：
2014-01-01
影响因子：
3.2
通讯作者：
Deng, Li
中科院分区：
文献类型：
--
作者：
Deng, Li
在这篇特邀论文中，我在APSIPA-2011全体概述会议上提出的关于同一主题的概述材料以及在同一会议上提出的教程材料[1]得到了扩展和更新，以包括深度学习的最新发展。本文从理论和应用两个方面对该技术进行了综述，并对该技术的未来发展方向进行了分析.本教程调查的目标是向APSIPA社区介绍深度学习或分层学习的新兴领域。深度学习是指一类机器学习技术，主要自2006年以来发展起来，其中分层架构中的非线性信息处理的许多阶段被用于模式分类和特征学习。在最近的文献中，它也与表征学习有关，表征学习涉及一个特征或概念的层次结构，其中较高层的概念是从较低层的概念定义的，而相同的较低层概念有助于定义较高层的概念。在本教程调查中，首先讨论了深度学习研究的简史。然后，开发了一个分类方案来分析和总结最近深度学习文献中报道的主要工作。使用这个方案，我提供了一个面向分类学的调查现有的深层架构和算法在文献中，并将它们分为三类：生成，歧视，和混合。三个有代表性的深度架构-深度自动编码器，深度堆叠网络及其推广到时域（递归网络），以及深度神经网络（用深度信念网络预训练），这三个类别中的每一个都有更详细的介绍。接下来，深度学习在信号和信息处理的广泛领域（包括音频/语音、图像/视觉、多模态、语言建模、自然语言处理和信息检索）中的选定应用进行了综述。最后，对深度学习的未来发展方向进行了讨论和分析。
In this invited paper, my overview material on the same topic as presented in the plenary overview session of APSIPA-2011 and the tutorial material presented in the same conference [1] are expanded and updated to include more recent developments in deep learning. The previous and the updatedmaterials cover both theory and applications, and analyze its future directions. The goal of this tutorial survey is to introduce the emerging area of deep learning or hierarchical learning to the APSIPA community. Deep learning refers to a class of machine learning techniques, developed largely since 2006, where many stages of non-linear information processing in hierarchical architectures are exploited for pattern classification and for feature learning. In the more recent literature, it is also connected to representation learning, which involves a hierarchy of features or concepts where higherlevel concepts are defined from lower-level ones and where the same lower-level concepts help to define higher-level ones. In this tutorial survey, a brief history of deep learning research is discussed first. Then, a classificatory scheme is developed to analyze and summarize major work reported in the recent deep learning literature. Using this scheme, I provide a taxonomy-oriented survey on the existing deep architectures and algorithms in the literature, and categorize them into three classes: generative, discriminative, and hybrid. Three representative deep architectures-deep autoencoders, deep stacking networks with their generalization to the temporal domain (recurrent networks), and deep neural networks (pretrained with deep belief networks) one in each of the three classes, are presented in more detail. Next, selected applications of deep learning are reviewed in broad areas of signal and information processing including audio/ speech, image/vision, multimodality, language modeling, natural language processing, and information retrieval. Finally, future directions of deep learning are discussed and analyzed.