A Constrained Optimization Approach to Preserving Prior Knowledge in Neural-Network Modeling and Control of Dynamical Systems

提案号：eccs -0823945提案题目：神经网络建模和控制中保留先验知识的约束优化方法名称：Ferrari， SilviaPI机构：Duke university本研究的目的是开发和实现人工神经网络中记忆和遗忘的统一理论。通过本研究开发的新型学习算法将消除非线性和全连接神经网络中的干扰和突变干扰，从而增强其在许多工程应用中的适用性。该方法是通过约束反向传播方法来制定学习，该方法优化受长期记忆约束的神经网络性能，这些约束可能会随着时间的推移通过惩罚函数或拉格朗日乘数而恶化。本研究的智力优势在于提出了一种结合约束优化理论和经典反向传播的约束反向传播方法。新开发的邻接误差梯度和代数训练形式一起允许高效和有效地制定约束反向传播，同时也利用现有的人工神经网络算法，如Levenberg-Marquardt和弹性反向传播。更广泛的影响提议的活动将增强在线自适应神经网络在广泛的复杂科学和工程问题中的适用性和有效性，即函数逼近，微分方程的解，系统识别和控制。约束反向传播理论和算法将在数据同化问题上实现，这将通过及时预测环境变化和城市污染物的扩散而造福社会，并在自适应双重控制上实现，这将产生容错和损伤的飞行控制系统，并使有人驾驶的飞机更安全，更容易飞行。此外，为了教育和传播的目的，还将通过使用图形用户界面的机器人技术和猎雷的基准问题进行示范。这种方法已经被证明是成功的，在杜克大学和教堂山公立学校的K-12学生以及当地小型工业之间创造了积极的协同效应和合作。

Proposal Number: ECCS-0823945Proposal Title: A Constrained Optimization Approach to Preserving Prior Knowledge in Neural-Network Modeling and Control of Dynamical SystemsPI Name: Ferrari, SilviaPI Institution: Duke UniversityThe objective of this research is to develop and implement a unified theory for memory and forgetting in artificial neural networks. The novel learning algorithms developed through this research will eliminate interference and catastrophic interference in nonlinear and fully-connected neural networks, thereby enhancing their applicability in a number of engineering applications. The approach is to formulate learning through a constrained backpropagation approach that optimizes the neural network performance subject to long-term memory constraints, which may be deteriorated over time via a penalty function or Lagrange multipliers.Intellectual MeritThe intellectual merit of the proposed research is the development of a novel constrained backpropagation approach that combines constrained optimization theory and classical backpropagation. The newly developed adjoined error gradient and algebraic training formalisms together allow to formulate constrained backpropagation efficiently and effectively, while also exploiting existing artificial neural networks algorithms, such as Levenberg-Marquardt and resilient backpropagation.Broader ImpactThe proposed activity will enhance the applicability and effectiveness of on-line adaptive neural networks in a broad spectrum of complex science and engineering problems, namely, function approximation, solution of differential equations, system identification, and control. The constrained-backpropagation theory and algorithms will be implemented on data-assimilation problems, which will benefit society by producing timely predictions about environmental change and dispersion of urban pollutants, and on adaptive dual control, which will produce flight control systems that are fault and damage-tolerant, and make piloted airplanes safer and easier to fly. Also, they will be demonstrated through benchmark problems in robotics and mine hunting using Graphical User Interfaces, for educational and dissemination purposes. This approach has already been proven successful at creating positive synergies and collaborations between Duke University and K-12 students from the Chapel Hill (NC) public schools, as well as small local industries.