CPS: Small: Data-Driven Modeling and Control of Human-Cyber-Physical Systems with Extended-Reality-Assisted Interfaces

人类-网络-物理系统（Human-cyber-physical systems，h-CPS）是一种交互式工程系统，它与一个或多个人类进行协作或交互，以利用人类和自主技术的互补优势。医疗设备、机器人辅助系统、远程操作、半自主系统和其他技术辅助应用都是h-CPS的例子。由于人类操作与h-CPS中的网络和物理过程密切相关，因此h-CPS分析和设计面临新的技术挑战，特别是在建模复杂的人类行为，实现有效的人机交互以及开发可靠和高性能的控制器方面。此外，如在未来的h-CPS中所设想的，从丰富的感测模态、建模、交互和控制程序中获得的足够质量和数量的大量数据正在从基于模型转变为数据驱动，并且预计会出现新的挑战，例如数据驱动方法的可信度和学习效率。该项目的目标是通过开发一个整体的数据驱动的设计框架，解决当今的主要障碍，应用数据驱动的方法建模，交互和控制的h-CPS的数据驱动的建模和控制的独特挑战。教育和推广活动很好地融入了研究，包括CPS劳动力培训，跨学科研究和课程开发，以及K-12 STEM推广活动。设计的活动具有独特的定位，以吸引代表性不足的群体的成员，重点是提高联邦，州和地方CPS劳动力的多样性。该项目将使数据驱动的建模和控制方法，如神经网络，强化学习，和基于模型的方法，如混合系统和模型预测控制的协同集成。该项目还将探索延展实境（XR）在构建人机界面方面的优势，以实现人类用户、机器和环境之间的有效沟通和互动。具体而言，该项目的研究活动将导致h-CPS的新发现：（1）将开发一种新的混合学习架构，用于人类行为的数据驱动建模，以准确捕获人类机动动力学和人类决策过程，（2）将开发X射线辅助界面，与半或全自主系统交互，以增强控制能力和性能，以及（3）将开发基于模型和数据驱动的控制协同设计框架，以实现人在工厂场景中的适应性和可靠控制，并将开发外环和内环控制器的协同设计框架，以帮助人类用户增强人类控制能力。验证实验将使用带有XR接口的h-CPS测试平台进行，例如眼动跟踪、基于凝视的控制和3D无人机远程操作系统。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Human-cyber-physical systems (h-CPS) are interactive engineered systems that collaborate or interact with one or more human beings to leverage the complementary strengths of both human and autonomy technologies. Medical devices, robot assistive systems, teleoperation, semi-autonomous systems, and other technology-assisted applications are all examples of h-CPS. Because human operations are deeply intertwined with cyber and physical processes in h-CPS, new technical challenges for h-CPS analysis and design emerge, particularly in modeling complex human behaviors, enabling effective human-machine interactions, and developing reliable and high-performance controllers. Furthermore, as envisioned in future h-CPS subject to a large amount of data of adequate quality and quantity available from rich sensing modalities, modeling, interaction, and control procedures are shifting from model-based to data-driven, and new challenges such as trustworthiness and learning efficiency of data-driven methods are expected to arise. This project targets the unique challenges of data-driven modeling and control of h-CPS by developing a holistic data-driven design framework, accounting for addressing today’s major barriers to apply data-driven approaches for modeling, interaction, and control of h-CPS. Educational and outreach activities are well-integrated into the research and include CPS workforce training, interdisciplinary research and curriculum development, and K-12 STEM outreach activities. The designed activities are uniquely positioned to attract members of underrepresented groups with a focus to enhance the diversity of the federal, state, and local CPS workforce.This project will enable the synergistic integration of data-driven modeling and control methods such as neural networks, reinforcement learning, and model-based methods such as hybrid systems, and model predictive control. This project will also explore the benefits of extended reality (XR) in building human-machine interfaces for effective communication and interaction between human users, machines, and environments. Specifically, the research activities in this project will lead to new discoveries of h-CPS in that: (1) a new hybrid learning architecture will be developed for data-driven modeling of human behaviors to accurately capture both human maneuver dynamics and human decision-making processes, (2) XR-assisted interfaces will be developed to interact with semi-or fully-autonomous systems to enhance control capability and performance, and (3) a model-based and data-driven control co-design framework will be developed to enable adaptable and reliable control in human-in-the-plant scenarios, and a co-design framework of outer-loop and inner-loop controllers will be developed to assist human users to augment human control capability. Validation experiments will be conducted using h-CPS testbeds with XR interfaces, such as eye-tracking, gaze-based control, and 3D drone teleoperation systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.