Real-time speech recognition and model selection via recurrent neural networks

我们通过深入研究学习的理论基础，对这篇报告的主题进行了研究。在第一年，我们开发了一个非常简单的递归神经网络（VSRN）架构，它是一个三层网络，在隐藏层中只包含自环递归连接。网络动力学解释了循环连接的作用，其功能将通过学习有限的例子（如哺乳动物的动作）来获得。通过学习过程，发现了哺乳动物听觉系统中观察到的一些特征功能，这些特征功能是由网络自动获得的。在第二年，我们主要研究了网络如何学习的理论框架，提出了一种分析泛化性能的新方法。为了实现这一点，我们进行了学习和假设检验的比较，这导致了正则插值维的新概念和产生不良假设的不良学习算法。这将学习和假设检验结合在一个共同的观点中，这样假设检验不等式的基础可以直接用于估计训练样本上的不良假设。规则插补尺寸不大于可修改系统参数的个数。我们分析了PAC学习模型和平均情况下的不良学习算法，以获得比VC维更明确的正则插值维的学习曲线和样本复杂度的界限。将所得结果推广到Gibbs算法和不一致学习等算法中，得到了学习曲线和样本复杂度的明确界。

We performed the study on the theme of this report by intensively investigating the theoretical base of learning. In the first year we developed a very simple recurrent neural network (VSRN) architecture which is a three-layr network and contains only self-loop recurrent connections in the hidden layr. The role of the recurrent connection is explained by the network dynamics and its function will be acquired by learning from finite examples like a mamalian action. Through the learning process some characteristic functions observed in the mamalian auditory systems are founed automatically acquired by the network. In the second year we investigated mainly the theoretical framework of how our network can learn well by proposing a new method for analysing the generalization performance. To achieve this, we undertake a comparison of learning and hypothesis testing, which leads to a novel notion of regular interpolation dimension and an ill-disposed learning algorithm that produces ill-disposed hypotheses. This unites the learning and the hypothesis testing in a common viewpoint such that the base of hypothesis testing inequalities can be directly used for estimating ill-disposed hypotheses on training examples. The regular interpolation dimension is no greater than the number of modifiable system parameters. We analyze the ill-disposed learning algorithm both in the PAC learning model and in an average-case setting to obtain more explicit bounds on learning curves and sample complexity in terms of the regular interpolation dimension, than those in terms of the VC dimension. The results are applied and extended to the other algorithm such as a Gibbs algorithm and the inconsistent learning to obtain explicit bound of the learning curves and sample complexity.