Collaborative Research: DARE: A Personalized Assistive Robotic System that assesses Cognitive Fatigue in Persons with Paralysis

随着机器人技术和人工智能的进步，辅助机器人系统有可能为脊髓损伤（SCI）患者提供支持和护理。随着机器人变得越来越普遍，比如今天的移动的手机，辅助机器人可以在帮助家中的残疾人、提高独立性和日常生活质量方面发挥重要作用。例如，机器人可以帮助有运动障碍的个体执行任务，例如准备午餐。目前的研究重点是确保在工业环境中安全的人机合作，这将不会威胁或损害人类队友的身体健康。然而，有有限的研究了解的认知或精神状态的人谁与辅助机器人每天进行日常生活活动的合作。这对于开发有效、直观和个性化的人机合作非常重要。该项目的目标是设计和开发一个端到端的个性化辅助机器人系统，称为iRCSA（智能机器人合作安全援助），以识别，评估和响应人类在人机合作过程中的认知疲劳。该系统的重点是人类-机器人合作任务，其中患有SCI的人类和机器人在日常任务期间合作（例如，烹饪）。有SCI经验的学生将参与项目的每个阶段，以确保拟议系统的可接受性和可用性。除了这项研究对改善残疾人生活独立性的重大影响外，该项目还包括为K-12学生开发新的辅助技术大学课程和暑期学校课程，以便学生获得机器人和辅助技术方面的知识，以便他们在科学，技术，工程和数学（STEM）方面的未来研究。 该项目将开发一个端到端的在线认知疲劳评估框架，作为拟议的人-机器人合作（HRC）系统的一部分，这构成了在适应中有效整合人类成分的能力的突破（个性化）的辅助机器人，并通过这三个重点解决关键的人权委员会的挑战：（a）开发一种新的人-技术系统，利用多模态数据进行在线认知疲劳评估;（B）开发一种适应性机器人系统，根据认知疲劳评估进行个性化互动;以及（c）HRC场景的实验测试台，以实现HRC、机器/深度学习和机器人系统计算进步。除了检测用户的认知疲劳之外，HRC系统还评估认知疲劳严重程度的水平，以便可以设计新的方法来基于用户的认知疲劳状态提供个性化的机器人交互。一个参与性的行动研究方法将遵循涉及运动障碍的学生在整个项目。系统评价将确定系统的优点和缺点，以改善瘫痪患者的HRC，并评估系统的可用性、可靠性和用户体验。除了辅助机器人应用之外，该项目还将推进多模态机器学习的研究，以便在需要机器人和人类合作的多个领域进行智能在线评估和控制。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

With the advancements in robotics and artificial intelligence, assistive robotic systems have the potential to provide support and care to people with Spinal Cord Injury (SCI). As robots become more widespread, like today’s mobile phones, assistive robots can play a significant role in assisting persons with disabilities at home, improving independence and everyday quality of life. For example, a robot may assist an individual with motor impairments to perform a task, such as preparing lunch. Current research focuses on ensuring safe human-robot cooperation in industrial environments, which will not threaten or harm the physical health of the human teammate. However, there is limited research on understanding the cognitive or mental state of a human who cooperates with an assistive robot daily for performing activities of daily living. This is important for developing effective, intuitive, and personalized human-robot cooperation. The objective of this project is to design and develop an end-to-end personalized assistive robotic system, called iRCSA (Intelligent Robotic Cooperation for Safe Assistance), to recognize, assess, and respond to a human’s cognitive fatigue during human-robot cooperation. The focus of the system is on human-robot cooperative tasks where a human with SCI and a robot cooperate during daily tasks (e.g., cooking). Students who have experienced SCI will be involved in every stage of the project, to ensure the acceptability and usability of the proposed system. In addition to the significant impact of this research on the improvement of life independence for persons with disabilities, the project includes the development of new university courses for assistive technologies and summer school programs for K-12 students, so that students gain knowledge on robotics and assistive technologies for their prospective studies in Science, Technology, Engineering, and Math (STEM). This project will develop an end-to-end framework for online cognitive fatigue assessment as part of the proposed Human-Robot Cooperation (HRC) system, which constitutes a breakthrough in the ability to effectively integrate the human component in the adaptation (personalization) of assistive robots and address key HRC challenges through these three thrusts: (a) The development of a novel human-technology system for online cognitive fatigue assessment using multimodal data; (b) the development of an adaptive robotic system for personalized interaction based on cognitive fatigue assessment; and (c) an experimental testbed of HRC scenarios to enable HRC, machine/deep learning, and robotic systems computational advances. In addition to detecting a user’s cognitive fatigue, the HRC system assesses the level of cognitive fatigue severity so that new methodologies can be designed to provide personalized robotic interactions based on the user’s state of cognitive fatigue. A participatory action research approach will be followed involving students with motor impairments throughout the project. Evaluation of the system will identify system benefits and weaknesses to improve HRC for persons with paralysis, as well as assess system usability, reliability, and user experience. Beyond the assistive robotic applications, the project will advance research in multimodal machine learning for intelligent online assessment and control in several fields where teaming between robots and humans is required.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.