Recurrent Deep Learning Machines

这项研究的目的是开发一种新的深度学习机器范式-那些具有反馈结构的机器。反馈将来自相同或更高层的其他计算节点的相邻或更大接受字段中包含的当前或过去信息带给计算节点，以形成更好的局部表示或特征。这些信息是处理动态数据和最大限度地提高对静态数据的概括能力所必需的。该研究的方法是选择或设计深度和递归结构，开发生成性和区分性学习技术，并将凸化训练标准的风险规避方法整合到训练递归深度学习机中。智能优点：递归神经网络在涉及动态数据的应用中是不可替代的，甚至在静态数据上也从根本上优于前馈网络。然而，培训经常性网络的困难扼杀了对它们的发展和了解。这项拟议的研究有望帮助消除这一困难，发挥递归神经网络的全部功能，并提高人们对神经网络的兴趣，这些神经网络在最近几年不幸地不受欢迎。更广泛的影响：经常性深度学习机具有强大的静态和动态分类和回归能力，包括图像和视频识别、分析和压缩；非线性系统识别/控制；信号处理/过滤；以及关键系统健康/故障监测/检测。因此，拟议的工作将对医疗仪器、计算机/机器人/信息技术、无线通信、国防和国土安全做出重大贡献。循环深度学习机将成为工程和计算机科学研究生教育的重要组成部分

The objective of this research is to develop a new paradigm of deep learning machine - those with a feedback structure. Feedbacks bring to computing nodes current or past information contained in neighboring or larger receptive fields of other computing nodes from the same or higher layers for forming better local representations or features. Such information is required for processing dynamical data and for maximizing generalization capabilities on static data. The approach of this research is to select or design deep and recurrent architectures, develop generative and discriminative learning techniques, and integrating the risk-averting method of convexifying training criteria into training recurrent deep learning machines. Intellectual Merit: Recurrent neural networks are irreplaceable for applications involving dynamical data and are fundamentally better than feedforward networks even on static data. However, difficulty in training recurrent networks has stifled development and understanding of them. The proposed research is expected to help remove this difficulty, bring forth the full power of recurrent neural networks, and boost interests in neural networks in general, which have unfortunately and undeservedly fallen out of favor in recent years. Broader Impact: Recurrent deep learning machines are powerful for static and dynamical classification and regression, including image and video recognition, analysis and compression; nonlinear system identification/control; signal processing/filtering; and critical system health/fault monitoring/detection. Therefore, the proposed work will contribute greatly to medical instrumentation, computer/robot/information technology, wireless telecommunication, national defense, and homeland security. Recurrent deep learning machines will ecome an important component in the graduate education in engineering and computer science