Collaborative Research: CPS: Medium: Physics-Model-Based Neural Networks Redesign for CPS Learning and Control

深度神经网络（DNN）支持的网络物理系统（CPS）为无人机和自动驾驶汽车等许多行业带来了巨大的变革。然而，当前一代的深度神经网络不能提供安全保证所必需的可分析行为和可验证属性。纯粹数据驱动的深度神经网络的这一关键缺陷有时会导致灾难性后果，例如与自动驾驶和驾驶辅助技术相关的车辆碰撞。另一方面，基于物理模型的工程方法提供了可分析的行为和可验证的属性，但与深度神经网络系统的性能不匹配。这些考虑因素激发了本项目的工作，该项目旨在重新设计基于物理模型的神经网络（NN），从而产生hhy -DNN：一种混合自校正物理增强的DNN框架。hhy - dnn将提供dnn的性能以及物理模型的可分析性和可验证性，从而为可验证安全的自动驾驶汽车、无人机和其他CPS系统提供基础。此外，hhy - dnn将从根本上推进深度学习和鲁棒控制的集成，使安全和时间关键型CPS能够在不可预见的动态环境中安全高效地运行。hcy - dnn将在重新设计NN架构方面进行三项创新：(i)对NN输入进行物理增强，以直接捕获难以学习的物理量并嵌入泰勒级数；（ii）物理指导的神经网络编辑，例如删除独立物理变量之间的联系，或在某些物理变量之间的联系上固定权重，以保持已知的物理特性，例如在守恒定律中；（iii）基于时间-频率表示滤波的激活，用于滤除具有动态频率分布的噪声。新颖的架构重新设计将赋予hhy - dnn四个目标能力：1)可控和可证明的模型精度；2)最大限度地避免虚假相关性；3)严格遵守物理知识；4)自动校正不安全的控制命令。最后，任何DNN的安全认证都将是一个长期的挑战。因此，为了保证在动态和不可预见的环境中安全稳定地运行，hhy - dnn必须有一个简单但经过验证的备份控制器。为了实现这一目标，研究团队将把hcy - dnn与自适应模型自适应控制（AMAC）框架相结合，其核心新颖之处在于通过稀疏回归快速准确地学习非线性模型，以实现基于模型的鲁棒控制。hly - dnn和AMAC是互补的，在安全状态周期内，它们将在不同规模的系统性能和功能上进行交互，并由Simplex软件架构提供支持，Simplex软件架构是一种知名的实时软件技术，可以容忍故障并允许在线控制系统升级。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deep Neural Networks (DNN) enabled Cyber-Physical Systems (CPS) hold great promise for revolutionizing many industries, such as drones and self-driving cars. However, the current generation of DNN cannot provide analyzable behaviors and verifiable properties that are necessary for safety assurance. This critical flaw in purely data-driven DNN sometimes leads to catastrophic consequences, such as vehicle crashes linked to self-driving and driver-assistance technologies. On the other hand, physics-model-based engineering methods provide analyzable behaviors and verifiable properties, but do not match the performance of DNN systems. These considerations motivate the work in this project which aims at physics-model-based neural networks (NN) redesign, yielding HyPhy-DNN: a hybrid self-correcting physics-enhanced DNN framework. HyPhy-DNN will provide the performance of DNNs and the analyzability and verifiability of physical models, thus providing a foundation for verifiably safe self-driving cars, drones, and other CPS systems. Moreover, the HyPhy-DNN will fundamentally advance the integration of deep learning and robust control to enable safety- and time-critical CPS to safely operate with high performance in unforeseen and dynamic environments.The HyPhy-DNN will make three innovations in redesigning NN architecture: (i) Physics augmentations of NN inputs for directly capturing hard-to-learn physical quantities and embedding Taylor series; (ii) Physics-guided neural network editing, such as removing links between independent physics variables or fixed weights on links between certain physics variables to maintain the known physics identity such as in conservation laws; and (iii) Time-frequency-representation filtering-based activations for filtering out noise having dynamic frequency distribution. The novel architectural redesigns will empower the HyPhy-DNN with four targeted capabilities: 1) controllable and provable model accuracy; 2) maximum avoidance of spurious correlations; 3) strict compliance with physics knowledge; and 4) automatic correction of unsafe control commands. Finally, the safety certification of any DNN will be a long-term challenge. Hence, the HyPhy-DNN shall have a simple but verified backup controller for guaranteeing safe and stable operation in dynamic and unforeseen environments. To achieve this, the research team will integrate HyPhy-DNN with an adaptive-model-adaptive-control (AMAC) framework, the core novelty of which lies in fast and accurate nonlinear model learning via sparse regression for model-based robust control. The HyPhy-DNN and AMAC are complementary and will be interactive at different scales of system performance and functionalities during the safety-status-cycle, supported by the Simplex software architecture, a well-known real-time software technology that tolerates faults and allows online control system upgrades.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.