CPS: Small: Data-Driven Modeling and Control of Human-Cyber-Physical Systems with Extended-Reality-Assisted Interfaces

CPS：小型：具有扩展现实辅助接口的人类网络物理系统的数据驱动建模和控制

• 批准号: 2223035
• 负责人: Weiming Xiang
• 金额: $ 49.9万
• 依托单位: AUGUSTA UNIVERSITY RESEARCH INSTITUTE, INC.
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2223035&HistoricalAwards=false
• 关键词: CPS; Small; Data; Driven; Modeling

Human-cyber-physical systems (h-CPS) are interactive engineered systems that collaborate or interact with one or more human beings to leverage the complementary strengths of both human and autonomy technologies. Medical devices, robot assistive systems, teleoperation, semi-autonomous systems, and other technology-assisted applications are all examples of h-CPS. Because human operations are deeply intertwined with cyber and physical processes in h-CPS, new technical challenges for h-CPS analysis and design emerge, particularly in modeling complex human behaviors, enabling effective human-machine interactions, and developing reliable and high-performance controllers. Furthermore, as envisioned in future h-CPS subject to a large amount of data of adequate quality and quantity available from rich sensing modalities, modeling, interaction, and control procedures are shifting from model-based to data-driven, and new challenges such as trustworthiness and learning efficiency of data-driven methods are expected to arise. This project targets the unique challenges of data-driven modeling and control of h-CPS by developing a holistic data-driven design framework, accounting for addressing today’s major barriers to apply data-driven approaches for modeling, interaction, and control of h-CPS. Educational and outreach activities are well-integrated into the research and include CPS workforce training, interdisciplinary research and curriculum development, and K-12 STEM outreach activities. The designed activities are uniquely positioned to attract members of underrepresented groups with a focus to enhance the diversity of the federal, state, and local CPS workforce.This project will enable the synergistic integration of data-driven modeling and control methods such as neural networks, reinforcement learning, and model-based methods such as hybrid systems, and model predictive control. This project will also explore the benefits of extended reality (XR) in building human-machine interfaces for effective communication and interaction between human users, machines, and environments. Specifically, the research activities in this project will lead to new discoveries of h-CPS in that: (1) a new hybrid learning architecture will be developed for data-driven modeling of human behaviors to accurately capture both human maneuver dynamics and human decision-making processes, (2) XR-assisted interfaces will be developed to interact with semi-or fully-autonomous systems to enhance control capability and performance, and (3) a model-based and data-driven control co-design framework will be developed to enable adaptable and reliable control in human-in-the-plant scenarios, and a co-design framework of outer-loop and inner-loop controllers will be developed to assist human users to augment human control capability. Validation experiments will be conducted using h-CPS testbeds with XR interfaces, such as eye-tracking, gaze-based control, and 3D drone teleoperation systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人-网络-物理系统（h-CPS）是一种交互式工程系统，它与一个或多个人类协作或交互，以利用人类和自主技术的互补优势。医疗设备、机器人辅助系统、远程操作、半自主系统和其他技术辅助应用都是h-CPS的例子。由于人类操作与h-CPS中的网络和物理过程深深交织在一起，因此出现了h-CPS分析和设计的新技术挑战，特别是在复杂的人类行为建模，实现有效的人机交互以及开发可靠和高性能控制器方面。此外，正如未来所设想的那样，由于丰富的传感模式提供了大量足够质量和数量的数据，建模、交互和控制程序正在从基于模型的转向数据驱动的，并且预计会出现数据驱动方法的可信度和学习效率等新挑战。该项目通过开发一个整体的数据驱动设计框架，针对h-CPS数据驱动建模和控制的独特挑战，解决了当今应用数据驱动方法进行h-CPS建模、交互和控制的主要障碍。教育和推广活动很好地融入了研究，包括CPS劳动力培训，跨学科研究和课程开发，以及K-12 STEM推广活动。设计的活动具有独特的定位，以吸引代表性不足的群体的成员，重点是增强联邦、州和当地CPS劳动力的多样性。该项目将实现数据驱动建模和控制方法的协同集成，如神经网络、强化学习、基于模型的方法，如混合系统和模型预测控制。该项目还将探索扩展现实（XR）在构建人机界面方面的好处，以便在人类用户、机器和环境之间进行有效的通信和交互。具体而言，本项目的研究活动将导致h-CPS的新发现：(1)将开发一种新的混合学习架构，用于数据驱动的人类行为建模，以准确捕获人类机动动力学和人类决策过程；(2)将开发xr辅助接口，与半或全自动系统交互，以增强控制能力和性能；(3)将开发一种基于模型和数据驱动的控制协同设计框架，以实现人类在工厂场景中的适应性和可靠控制。并将开发一个外环和内环控制器的协同设计框架，以帮助人类用户增强人类的控制能力。验证实验将使用带有XR接口的h-CPS试验台进行，如眼球追踪、基于凝视的控制和3D无人机远程操作系统。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Computationally efficient neural hybrid automaton framework for learning complex dynamics

• DOI: 10.1016/j.neucom.2023.126879
• 发表时间: 2023-10
• 期刊: Neurocomputing
• 影响因子: 6
• 作者: Tao Wang;Yejiang Yang;Weiming Xiang

Maximum output discrepancy computation for convolutional neural network compression

• DOI: 10.1016/j.ins.2024.120367
• 发表时间: 2024-04
• 期刊: Inf. Sci.
• 作者: Zihao Mo;Weiming Xiang

A Data-Driven Hybrid Automaton Framework to Modeling Complex Dynamical Systems

• DOI: 10.1109/icit58465.2023.10143031
• 发表时间: 2023-04
• 期刊: 2023 IEEE International Conference on Industrial Technology (ICIT)
• 作者: Yejiang Yang;Zihao Mo;Weiming Xiang

Modeling Dynamical Systems with Neural Hybrid System Framework via Maximum Entropy Approach

• DOI: 10.23919/acc55779.2023.10155820
• 发表时间: 2023-05
• 期刊: 2023 American Control Conference (ACC)
• 作者: Yejiang Yang;Weiming Xiang

Guaranteed Quantization Error Computation for Neural Network Model Compression

神经网络模型压缩的保证量化误差计算

• DOI: 10.1109/icit58465.2023.10143141
• 发表时间: 2023
• 期刊: 2023 IEEE International Conference on Industrial Technology (ICIT
• 作者: Cooke, Wesley;Mo, Zihao;Xiang, Weiming
• 通讯作者: Xiang, Weiming