Collaborative Research: DARE: A Personalized Assistive Robotic System that assesses Cognitive Fatigue in Persons with Paralysis

合作研究：DARE：一种评估瘫痪者认知疲劳的个性化辅助机器人系统

• 批准号: 2226164
• 负责人: Fillia Makedon
• 金额: $ 21.83万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2026-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2226164&HistoricalAwards=false
• 关键词: Collaborative; Research; DARE; Personalized; Assistive

With the advancements in robotics and artificial intelligence, assistive robotic systems have the potential to provide support and care to people with Spinal Cord Injury (SCI). As robots become more widespread, like today’s mobile phones, assistive robots can play a significant role in assisting persons with disabilities at home, improving independence and everyday quality of life. For example, a robot may assist an individual with motor impairments to perform a task, such as preparing lunch. Current research focuses on ensuring safe human-robot cooperation in industrial environments, which will not threaten or harm the physical health of the human teammate. However, there is limited research on understanding the cognitive or mental state of a human who cooperates with an assistive robot daily for performing activities of daily living. This is important for developing effective, intuitive, and personalized human-robot cooperation. The objective of this project is to design and develop an end-to-end personalized assistive robotic system, called iRCSA (Intelligent Robotic Cooperation for Safe Assistance), to recognize, assess, and respond to a human’s cognitive fatigue during human-robot cooperation. The focus of the system is on human-robot cooperative tasks where a human with SCI and a robot cooperate during daily tasks (e.g., cooking). Students who have experienced SCI will be involved in every stage of the project, to ensure the acceptability and usability of the proposed system. In addition to the significant impact of this research on the improvement of life independence for persons with disabilities, the project includes the development of new university courses for assistive technologies and summer school programs for K-12 students, so that students gain knowledge on robotics and assistive technologies for their prospective studies in Science, Technology, Engineering, and Math (STEM). This project will develop an end-to-end framework for online cognitive fatigue assessment as part of the proposed Human-Robot Cooperation (HRC) system, which constitutes a breakthrough in the ability to effectively integrate the human component in the adaptation (personalization) of assistive robots and address key HRC challenges through these three thrusts: (a) The development of a novel human-technology system for online cognitive fatigue assessment using multimodal data; (b) the development of an adaptive robotic system for personalized interaction based on cognitive fatigue assessment; and (c) an experimental testbed of HRC scenarios to enable HRC, machine/deep learning, and robotic systems computational advances. In addition to detecting a user’s cognitive fatigue, the HRC system assesses the level of cognitive fatigue severity so that new methodologies can be designed to provide personalized robotic interactions based on the user’s state of cognitive fatigue. A participatory action research approach will be followed involving students with motor impairments throughout the project. Evaluation of the system will identify system benefits and weaknesses to improve HRC for persons with paralysis, as well as assess system usability, reliability, and user experience. Beyond the assistive robotic applications, the project will advance research in multimodal machine learning for intelligent online assessment and control in several fields where teaming between robots and humans is required.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着机器人学和人工智能的进步，辅助机器人系统有可能为脊髓损伤(SCI)患者提供支持和护理。随着机器人变得越来越普遍，就像今天的移动电话一样，辅助机器人可以在帮助家庭残疾人、提高独立性和日常生活质量方面发挥重要作用。例如，机器人可以帮助有运动障碍的人执行一项任务，如准备午餐。目前的研究集中在确保工业环境中安全的人-机器人合作，而不会威胁或损害人类队友的身体健康。然而，关于理解每天与辅助机器人合作进行日常生活活动的人类的认知或精神状态的研究有限。这对于开发有效、直观和个性化的人-机器人协作非常重要。本项目的目标是设计和开发一个端到端的个性化辅助机器人系统，称为iRCSA(智能机器人安全协助合作)，以识别、评估和应对人与机器人合作过程中的认知疲劳。该系统的重点是人-机器人合作任务，即患有脊髓损伤的人和机器人在日常任务(如做饭)中合作。有过SCI经历的学生将参与到项目的每个阶段，以确保拟议系统的可接受性和可用性。除了这项研究对提高残疾人的生活独立性有重大影响外，该项目还包括为K-12学生开发新的辅助技术大学课程和暑期学校计划，以便学生在未来的科学、技术、工程和数学(STEM)学习中获得机器人和辅助技术的知识。该项目将开发在线认知疲劳评估的端到端框架，作为拟议的人-机器人合作系统的一部分，这是在有效地将人的因素纳入辅助机器人适应(个性化)方面的能力方面的突破，并通过以下三项努力应对人权委员会的关键挑战：(A)开发利用多模式数据进行在线认知疲劳评估的新型人-技术系统；(B)开发基于认知疲劳评估的个性化互动的自适应机器人系统；(C)人权委员会情景的实验试验台，以推动人权委员会、机器/深度学习和机器人系统的计算。除了检测用户的认知疲劳外，HRC系统还评估认知疲劳的严重程度，以便设计新的方法，根据用户的认知疲劳状态提供个性化的机器人交互。在整个项目中，将采用参与式行动研究方法，让运动障碍的学生参与进来。对该系统的评估将确定系统的优点和弱点，以改善瘫痪患者的人权状况，并评估系统的可用性、可靠性和用户体验。除了辅助机器人应用，该项目还将在几个需要机器人和人类合作的领域推进多模式机器学习的智能在线评估和控制的研究。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Assessment of Cognitive Fatigue from Gait Cycle Analysis

通过步态周期分析评估认知疲劳

• DOI: 10.3390/technologies11010018
• 发表时间: 2023
• 期刊: Technologies
• 影响因子: 3.6
• 作者: Pavel, Hamza Reza;Karim, Enamul;Jaiswal, Ashish;Acharya, Sneh;Nale, Gaurav;Theofanidis, Michail;Makedon, Fillia
• 通讯作者: Makedon, Fillia

An EEG-based Cognitive Fatigue Detection System

• DOI: 10.1145/3594806.3594848
• 发表时间: 2023-07
• 期刊: Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments
• 作者: Enamul Karim;Hamza Reza Pavel;Ashish Jaiswal;Mohammad Zaki Zadeh;Michail Theofanidis;G. Wylie;F. Makedon

A Smart Sensor Suit (SSS) to Assess Cognitive and Physical Fatigue with Machine Learning

通过机器学习评估认知和身体疲劳的智能传感器套装 (SSS)

• 发表时间: 2023
• 期刊: HCII 2023
• 作者: Jaiswal, Ashish;Zadeh, Mohammad Zaki;Hebri, Aref;Babu, Ashwin Ramesh;Makedon, Fillia.
• 通讯作者: Makedon, Fillia.