CPS: Small: Data-Driven Modeling and Control of Human-Cyber-Physical Systems with Extended-Reality-Assisted Interfaces

人-网络-物理系统(h-CP)是一种交互式工程系统，它与一个或多个人类协作或交互，以利用人类和自主技术的互补优势。医疗设备、机器人辅助系统、遥操作、半自动系统和其他技术辅助应用都是h-CP的例子。由于h-CPS中的人工操作与网络和物理过程深度交织在一起，h-CPS的分析和设计面临着新的技术挑战，特别是在建模复杂的人类行为、实现有效的人机交互以及开发可靠和高性能的控制器方面。此外，正如在未来的h-CP中所设想的那样，随着从丰富的传感模式获得的大量足够质量和数量的数据，建模、交互和控制程序正从基于模型转向数据驱动，预计将出现诸如数据驱动方法的可信性和学习效率等新的挑战。该项目针对h-CP的数据驱动建模和控制的独特挑战，开发了一个整体的数据驱动设计框架，解决了当今在h-CP的建模、交互和控制中应用数据驱动方法的主要障碍。教育和外展活动很好地融入了研究，包括CPS劳动力培训、跨学科研究和课程开发，以及K-12 STEM外展活动。设计的活动具有独特的定位，以吸引代表性不足的群体成员，重点是增强联邦、州和地方CPS工作人员的多样性。该项目将使数据驱动的建模和控制方法(如神经网络、强化学习)与基于模型的方法(如混合系统和模型预测控制)的协同集成成为可能。该项目还将探索扩展现实(XR)在构建人机界面方面的好处，以便在人类用户、机器和环境之间进行有效的通信和交互。具体地说，本项目中的研究活动将导致h-CP的新发现：(1)将开发一种新的混合学习体系结构，用于数据驱动的人类行为建模，以准确地捕捉人类动作动力学和人类决策过程，(2)将开发XR辅助接口，以与半自主或全自主系统交互，以增强控制能力和性能，以及(3)将开发基于模型和数据驱动的控制共同设计框架，以实现人在工厂场景中的自适应和可靠控制，并开发了外环和内环控制器的联合设计框架，以帮助人类用户增强人类的控制能力。验证实验将使用具有XR接口的h-CPS试验台进行，例如眼睛跟踪、基于凝视的控制和3D无人机远程操作系统。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Human-cyber-physical systems (h-CPS) are interactive engineered systems that collaborate or interact with one or more human beings to leverage the complementary strengths of both human and autonomy technologies. Medical devices, robot assistive systems, teleoperation, semi-autonomous systems, and other technology-assisted applications are all examples of h-CPS. Because human operations are deeply intertwined with cyber and physical processes in h-CPS, new technical challenges for h-CPS analysis and design emerge, particularly in modeling complex human behaviors, enabling effective human-machine interactions, and developing reliable and high-performance controllers. Furthermore, as envisioned in future h-CPS subject to a large amount of data of adequate quality and quantity available from rich sensing modalities, modeling, interaction, and control procedures are shifting from model-based to data-driven, and new challenges such as trustworthiness and learning efficiency of data-driven methods are expected to arise. This project targets the unique challenges of data-driven modeling and control of h-CPS by developing a holistic data-driven design framework, accounting for addressing today’s major barriers to apply data-driven approaches for modeling, interaction, and control of h-CPS. Educational and outreach activities are well-integrated into the research and include CPS workforce training, interdisciplinary research and curriculum development, and K-12 STEM outreach activities. The designed activities are uniquely positioned to attract members of underrepresented groups with a focus to enhance the diversity of the federal, state, and local CPS workforce.This project will enable the synergistic integration of data-driven modeling and control methods such as neural networks, reinforcement learning, and model-based methods such as hybrid systems, and model predictive control. This project will also explore the benefits of extended reality (XR) in building human-machine interfaces for effective communication and interaction between human users, machines, and environments. Specifically, the research activities in this project will lead to new discoveries of h-CPS in that: (1) a new hybrid learning architecture will be developed for data-driven modeling of human behaviors to accurately capture both human maneuver dynamics and human decision-making processes, (2) XR-assisted interfaces will be developed to interact with semi-or fully-autonomous systems to enhance control capability and performance, and (3) a model-based and data-driven control co-design framework will be developed to enable adaptable and reliable control in human-in-the-plant scenarios, and a co-design framework of outer-loop and inner-loop controllers will be developed to assist human users to augment human control capability. Validation experiments will be conducted using h-CPS testbeds with XR interfaces, such as eye-tracking, gaze-based control, and 3D drone teleoperation systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.