Real-time speech recognition and model selection via recurrent neural networks

我们围绕本报告的主题，深入研究了学习的理论基础。在第一年，我们开发了一个非常简单的递归神经网络（VSRN）架构，它是一个三层网络，在隐藏层中只包含自循环递归连接。循环连接的作用由网络动力学解释，其功能将通过从有限的例子中学习获得，就像一个哺乳动物的行动。通过学习过程，发现了一些在哺乳动物听觉系统中观察到的特征函数。在第二年，我们主要研究了我们的网络如何通过提出一种分析泛化性能的新方法来学习的理论框架。为了实现这一目标，我们进行了学习和假设检验的比较，这导致了一个新的概念，定期插值维数和病态的学习算法，产生病态的假设。这将学习和假设检验统一在一个共同的观点中，使得假设检验不等式的基础可以直接用于估计训练示例上的不良假设。常规插值维数不大于可修改的系统参数数。我们分析了PAC学习模型和平均情况下的病态学习算法，以获得更明确的边界上的学习曲线和样本的复杂性方面的定期插值维数，比那些在VC维。将所得结果应用于Gibbs算法和不一致学习等其他算法，得到了学习曲线和样本复杂度的显式界。

We performed the study on the theme of this report by intensively investigating the theoretical base of learning. In the first year we developed a very simple recurrent neural network (VSRN) architecture which is a three-layr network and contains only self-loop recurrent connections in the hidden layr. The role of the recurrent connection is explained by the network dynamics and its function will be acquired by learning from finite examples like a mamalian action. Through the learning process some characteristic functions observed in the mamalian auditory systems are founed automatically acquired by the network. In the second year we investigated mainly the theoretical framework of how our network can learn well by proposing a new method for analysing the generalization performance. To achieve this, we undertake a comparison of learning and hypothesis testing, which leads to a novel notion of regular interpolation dimension and an ill-disposed learning algorithm that produces ill-disposed hypotheses. This unites the learning and the hypothesis testing in a common viewpoint such that the base of hypothesis testing inequalities can be directly used for estimating ill-disposed hypotheses on training examples. The regular interpolation dimension is no greater than the number of modifiable system parameters. We analyze the ill-disposed learning algorithm both in the PAC learning model and in an average-case setting to obtain more explicit bounds on learning curves and sample complexity in terms of the regular interpolation dimension, than those in terms of the VC dimension. The results are applied and extended to the other algorithm such as a Gibbs algorithm and the inconsistent learning to obtain explicit bound of the learning curves and sample complexity.