Accelerated Vonvergence and Structure Determination of the Backpropagation Neural Network

神经网络可以识别和学习相关模式 输入数据集和相应的目标值之间的关系。 最广泛使用的神经网络架构，后- 传播网络，松散地模仿人类的学习过程， "学会"识别与输入和输出相关的模式 变量 这种蚊帐通过反复喂食来训练 输入数据以及相应的目标结果。 后 足够数量的训练迭代，网络学会 识别数据中的模式，并有效地创建 管理数据过程的内部模型。 网可 然后使用这个内部模型来预测新的输入 条件 反向传播神经网络的监督训练 网络通常是通过一个解决方案来实现的， 适当的优化问题。 随后，培训 非线性规划算法的影响 采用 经常使用的训练算法是delta 规则，这是一个最陡的下降导数，因此 在局部最小值附近表现出线性收敛率。 这导致了非常长的训练时间，通常在 几个小时或几天来解决实际问题。 在本项目中，PI计划：（1）加快培训 使用牛顿型算法的反向传播网络，（2） 确定网络结构通过使用奇异 解析hessian的值分解，（3）使用 最小生成网络的概念，以导出 线性元素，将提供性能下限 神经网络，以及（4）施加适当的界限（或 约束）以增强收敛。 他 希望这些结果将大大加快 训练神经网络。 两者的结构 将利用解析梯度和解析海森函数， 在反向传播算法的实现中， 并行计算机，从而进一步提高速度 起来 以这样的速度，将有可能解决 在合理的时间内解决与工业有关的难题 frame.

Neural nets can identify and learn correlative patterns between sets of input data and corresponding target values. The most widely used neural net architecture, the back- propagation net, loosely mimics the human learning process and "learns" to recognize patterns relating input and output variables. Such nets are trained by being repeatedly fed input data together with corresponding target outcomes. After a sufficient number of training iterations, the net learns to recognize patterns in the data and, effectively, creates an internal model of the process governing the data. The net can then use this internal model to make predictions for new input conditions. Supervised training of back-propagation neural networks is usually achieved through the solution of an appropriate optimization problem. Subsequently, training times are affected by the nonlinear programming algorithms used. The training algorithm that is often used is the delta rule, which is a steepest descent derivative and as such exhibits a linear rate of convergence around a local minimum. This results in very long training time, often on the order of hours or days for practical problems. In this project the PI plans to: (1) accelerate training of the back-propagation network using Newton type algorithms, (2) determine network structure through the use of the singular value decomposition of the analytic hessian, (3) use the concept of a Minimal Spanning Network to derive a network of linear elements that will provide a performance lower bound on the neural network, and (4) impose appropriate bounds (or constraints) on design variables to enhance convergence. He hopes that the results will significantly speed up the training of neural networks. The structure of both the analytic gradients and the analytic hessian will be exploited in an implementation of the back-propagation algorithm on parallel computers, resulting in further increases in speed up. With such speed up, it will be possible to tackle difficult industrially relevant problems in a reasonable time frame.