A Constrained Optimization Approach to Preserving Prior Knowledge in Neural-Network Modeling and Control of Dynamical Systems

提案编号：ECCS-0823945提案标题：在动态系统的神经网络建模和控制中保留先验知识的约束优化方法PI姓名： Ferrari，SilviaPI机构：杜克大学本研究的目的是发展和实现一个统一的理论，记忆和遗忘的人工神经网络。 通过这项研究开发的新型学习算法将消除非线性和全连接神经网络中的干扰和灾难性干扰，从而增强其在许多工程应用中的适用性。 该方法是制定学习通过约束反向传播的方法，优化神经网络的性能受到长期的记忆约束，这可能会随着时间的推移，通过一个惩罚函数或拉格朗日multiplers.Intellectual MeritThe智力的优点，提出的研究是一种新的约束反向传播的方法，结合约束优化理论和经典的反向传播的发展。 新开发的邻接误差梯度和代数训练形式主义一起允许制定约束反向传播高效和有效地，同时也利用现有的人工神经网络算法，如Levenberg-Marquardt和弹性反向传播。更广泛的影响拟议的活动将提高在线自适应神经网络在广泛的复杂科学和工程问题，即，函数逼近、微分方程解、系统辨识和控制。 约束反向传播理论和算法将用于数据同化问题，这将通过及时预测环境变化和城市污染物的扩散而造福社会，并用于自适应双重控制，这将产生容错和损伤容限的飞行控制系统，使有人驾驶的飞机更安全，更容易飞行。 此外，为了教育和传播目的，还将利用图形用户界面，通过机器人技术和扫雷方面的基准问题来演示。 事实证明，这种方法在杜克大学与查佩尔山（NC）公立学校的K-12学生以及当地小型企业之间创造了积极的协同效应和合作关系。

Proposal Number: ECCS-0823945Proposal Title: A Constrained Optimization Approach to Preserving Prior Knowledge in Neural-Network Modeling and Control of Dynamical SystemsPI Name: Ferrari, SilviaPI Institution: Duke UniversityThe objective of this research is to develop and implement a unified theory for memory and forgetting in artificial neural networks. The novel learning algorithms developed through this research will eliminate interference and catastrophic interference in nonlinear and fully-connected neural networks, thereby enhancing their applicability in a number of engineering applications. The approach is to formulate learning through a constrained backpropagation approach that optimizes the neural network performance subject to long-term memory constraints, which may be deteriorated over time via a penalty function or Lagrange multipliers.Intellectual MeritThe intellectual merit of the proposed research is the development of a novel constrained backpropagation approach that combines constrained optimization theory and classical backpropagation. The newly developed adjoined error gradient and algebraic training formalisms together allow to formulate constrained backpropagation efficiently and effectively, while also exploiting existing artificial neural networks algorithms, such as Levenberg-Marquardt and resilient backpropagation.Broader ImpactThe proposed activity will enhance the applicability and effectiveness of on-line adaptive neural networks in a broad spectrum of complex science and engineering problems, namely, function approximation, solution of differential equations, system identification, and control. The constrained-backpropagation theory and algorithms will be implemented on data-assimilation problems, which will benefit society by producing timely predictions about environmental change and dispersion of urban pollutants, and on adaptive dual control, which will produce flight control systems that are fault and damage-tolerant, and make piloted airplanes safer and easier to fly. Also, they will be demonstrated through benchmark problems in robotics and mine hunting using Graphical User Interfaces, for educational and dissemination purposes. This approach has already been proven successful at creating positive synergies and collaborations between Duke University and K-12 students from the Chapel Hill (NC) public schools, as well as small local industries.