A Constrained Optimization Approach to Preserving Prior Knowledge in Neural-Network Modeling and Control of Dynamical Systems

提案编号：ECCS-0823945提案标题：在动态系统的神经网络建模和控制中保留先验知识的约束优化方法PI名称：Ferrari，SilviaPI机构：杜克大学这项研究的目标是开发和实现人工神经网络中记忆和遗忘的统一理论。通过本研究开发的新型学习算法将消除非线性和全连接神经网络中的干扰和灾难性干扰，从而增强其在许多工程应用中的适用性。该方法是通过约束反向传播方法来描述学习，该方法在长期记忆约束下优化神经网络的性能，长期记忆约束可能会通过惩罚函数或拉格朗日乘数随着时间的推移而恶化。新开发的附加误差梯度和代数训练公式允许高效和有效地建立约束反向传播，同时还可以利用现有的人工神经网络算法，如Levenberg-MarQuardt和弹性反向传播。广泛影响建议的活动将增强在线自适应神经网络在广泛的复杂科学和工程问题中的适用性和有效性，即函数逼近、微分方程组的求解、系统辨识和控制。约束反向传播理论和算法将应用于数据同化问题和自适应双重控制问题，前者将通过产生对环境变化和城市污染物扩散的及时预测而造福社会，后者将产生故障和损伤容限的飞行控制系统，并使有人驾驶的飞机更安全、更容易飞行。此外，为了教育和传播的目的，将通过使用图形用户界面的机器人和猎雷方面的基准问题来演示它们。这种方法已经被证明成功地在杜克大学和来自教堂山(北卡罗来纳州)公立学校以及当地小型企业的K-12学生之间创造了积极的协同和合作。

Proposal Number: ECCS-0823945Proposal Title: A Constrained Optimization Approach to Preserving Prior Knowledge in Neural-Network Modeling and Control of Dynamical SystemsPI Name: Ferrari, SilviaPI Institution: Duke UniversityThe objective of this research is to develop and implement a unified theory for memory and forgetting in artificial neural networks. The novel learning algorithms developed through this research will eliminate interference and catastrophic interference in nonlinear and fully-connected neural networks, thereby enhancing their applicability in a number of engineering applications. The approach is to formulate learning through a constrained backpropagation approach that optimizes the neural network performance subject to long-term memory constraints, which may be deteriorated over time via a penalty function or Lagrange multipliers.Intellectual MeritThe intellectual merit of the proposed research is the development of a novel constrained backpropagation approach that combines constrained optimization theory and classical backpropagation. The newly developed adjoined error gradient and algebraic training formalisms together allow to formulate constrained backpropagation efficiently and effectively, while also exploiting existing artificial neural networks algorithms, such as Levenberg-Marquardt and resilient backpropagation.Broader ImpactThe proposed activity will enhance the applicability and effectiveness of on-line adaptive neural networks in a broad spectrum of complex science and engineering problems, namely, function approximation, solution of differential equations, system identification, and control. The constrained-backpropagation theory and algorithms will be implemented on data-assimilation problems, which will benefit society by producing timely predictions about environmental change and dispersion of urban pollutants, and on adaptive dual control, which will produce flight control systems that are fault and damage-tolerant, and make piloted airplanes safer and easier to fly. Also, they will be demonstrated through benchmark problems in robotics and mine hunting using Graphical User Interfaces, for educational and dissemination purposes. This approach has already been proven successful at creating positive synergies and collaborations between Duke University and K-12 students from the Chapel Hill (NC) public schools, as well as small local industries.