Collaborative Research: DARE: A Personalized Assistive Robotic System that assesses Cognitive Fatigue in Persons with Paralysis

随着机器人技术和人工智能的进步，辅助机器人系统有可能为脊髓损伤（SCI）患者提供支持和护理。随着机器人变得越来越普遍，就像今天的移动电话一样，辅助机器人可以在帮助家中残疾人，提高独立性和日常生活质量方面发挥重要作用。例如，机器人可以帮助有运动障碍的人完成一项任务，比如准备午餐。目前的研究重点是在工业环境下保证人机安全合作，不威胁或伤害人类队友的身体健康。然而，对于与辅助机器人合作进行日常生活活动的人的认知或精神状态的理解研究有限。这对于开发有效、直观和个性化的人机合作非常重要。该项目的目标是设计和开发一个端到端的个性化辅助机器人系统，称为iRCSA（智能机器人合作安全援助），以识别、评估和响应人类在人机合作过程中的认知疲劳。该系统的重点是人机合作任务，其中SCI患者和机器人在日常任务（例如烹饪）中合作。经历过SCI的学生将参与项目的每个阶段，以确保拟议系统的可接受性和可用性。除了这项研究对改善残疾人的生活独立性产生重大影响外，该项目还包括为K-12学生开发新的辅助技术大学课程和暑期学校课程，以便学生获得机器人和辅助技术的知识，为他们未来的科学，技术，工程和数学（STEM）学习。该项目将开发一个端到端框架，用于在线认知疲劳评估，作为拟议的人机合作（HRC）系统的一部分，这构成了在辅助机器人的适应（个性化）中有效整合人的组成部分的能力的突破，并通过以下三个重点解决关键的HRC挑战：(a)开发一种新的人类技术系统，用于使用多模态数据进行在线认知疲劳评估；(b)开发基于认知疲劳评估的个性化交互的自适应机器人系统；(c) HRC场景的实验测试平台，以实现HRC、机器/深度学习和机器人系统的计算进步。除了检测用户的认知疲劳之外，HRC系统还可以评估认知疲劳的严重程度，以便设计新的方法，根据用户的认知疲劳状态提供个性化的机器人交互。在整个项目中，将采用参与式行动研究方法，让有运动障碍的学生参与。对该系统的评估将确定系统的优点和缺点，以改善瘫痪患者的HRC，并评估系统的可用性、可靠性和用户体验。除了辅助机器人应用之外，该项目还将推进多模式机器学习的研究，以便在需要机器人和人类之间合作的几个领域进行智能在线评估和控制。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

With the advancements in robotics and artificial intelligence, assistive robotic systems have the potential to provide support and care to people with Spinal Cord Injury (SCI). As robots become more widespread, like today’s mobile phones, assistive robots can play a significant role in assisting persons with disabilities at home, improving independence and everyday quality of life. For example, a robot may assist an individual with motor impairments to perform a task, such as preparing lunch. Current research focuses on ensuring safe human-robot cooperation in industrial environments, which will not threaten or harm the physical health of the human teammate. However, there is limited research on understanding the cognitive or mental state of a human who cooperates with an assistive robot daily for performing activities of daily living. This is important for developing effective, intuitive, and personalized human-robot cooperation. The objective of this project is to design and develop an end-to-end personalized assistive robotic system, called iRCSA (Intelligent Robotic Cooperation for Safe Assistance), to recognize, assess, and respond to a human’s cognitive fatigue during human-robot cooperation. The focus of the system is on human-robot cooperative tasks where a human with SCI and a robot cooperate during daily tasks (e.g., cooking). Students who have experienced SCI will be involved in every stage of the project, to ensure the acceptability and usability of the proposed system. In addition to the significant impact of this research on the improvement of life independence for persons with disabilities, the project includes the development of new university courses for assistive technologies and summer school programs for K-12 students, so that students gain knowledge on robotics and assistive technologies for their prospective studies in Science, Technology, Engineering, and Math (STEM). This project will develop an end-to-end framework for online cognitive fatigue assessment as part of the proposed Human-Robot Cooperation (HRC) system, which constitutes a breakthrough in the ability to effectively integrate the human component in the adaptation (personalization) of assistive robots and address key HRC challenges through these three thrusts: (a) The development of a novel human-technology system for online cognitive fatigue assessment using multimodal data; (b) the development of an adaptive robotic system for personalized interaction based on cognitive fatigue assessment; and (c) an experimental testbed of HRC scenarios to enable HRC, machine/deep learning, and robotic systems computational advances. In addition to detecting a user’s cognitive fatigue, the HRC system assesses the level of cognitive fatigue severity so that new methodologies can be designed to provide personalized robotic interactions based on the user’s state of cognitive fatigue. A participatory action research approach will be followed involving students with motor impairments throughout the project. Evaluation of the system will identify system benefits and weaknesses to improve HRC for persons with paralysis, as well as assess system usability, reliability, and user experience. Beyond the assistive robotic applications, the project will advance research in multimodal machine learning for intelligent online assessment and control in several fields where teaming between robots and humans is required.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.