Investigation of User-Interface and Human-Robot Performance for Supernumerary Robots

多余机器人的用户界面和人机性能研究

• 批准号: 1934792
• 负责人: Sanjay Joshi
• 金额: $ 84.87万
• 依托单位: University of California-Davis
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1934792&HistoricalAwards=false
• 关键词: Investigation; User; Interface; Human; Robot

Supernumerary robots are assistive devices that may attach to the human body and be controlled to perform coordinated actions with a person's natural limbs. Supernumerary robots have potential to reduce risk and increase human capability in physically-demanding and/or dangerous workplace activities, including construction, agricultural applications, search-and-rescue, and outer space exploration. The research objective of this project is to advance a fundamental understanding of human-machine interaction within the context of real-time dexterous control of a "third arm" supernumerary robot. The project will develop a novel human-machine interface that will allow a human to use electromyographic activity (EMG) to command the robot in a closed-loop manner using multimodal sensory feedback derived from visual and supplemental vibrotactile information. The project team will use the new system in two sets of human subject experiments designed to explore the bi-directional adaptation that occurs within the human-robot dyad as the human learns to control the machine in a novel three-handed manipulation task, even as the robotic controller adapts to the changing signal properties of muscle EMGs as the human user fatigues. The project advances the National Science Foundation's mission to promote the progress of science, to advance national health, prosperity, and welfare, by advancing a fundamental understanding of co-adaptive human-machine partnerships. Broader impacts of this project include establishing a summer program for elementary and middle school female students to introduce principles of robotics, sensors, neuroscience, and human performance. The summer program is directly aligned with the National Science Foundation's goal to support activities designed to increase the participation of women and other underrepresented groups in science and technology.This project explores human-machine cooperation within the context of a "third arm" supernumerary robot. Two sets of human subject experiments are planned. The first establishes the feasibility of a novel, adaptive, EMG-based methodology for haptic-guided closed-loop control of a supernumerary robotic arm's end-effector. Several biosignals sensitive to cognitive state will be recorded, including pupillometry, galvanic skin response, heart rate variability, and electroencephalography (EEG). Subjects will perform a point-to-point trajectory task and a 3D version of the Fitts's Law task to assess control efficiency with and without vibrotactile performance feedback applied to the user's foot. The second explores the coupled dynamics of short-term human learning of a three-handed manipulation task that requires the user to coordinate motions of their two hands with those of the supernumerary robot to manipulate a novel three-element object to recreate specific desired poses. At the same time, online computational learning adaptations of the robot's grip force will be driven by perturbations of object kinematics and kinetics caused by the user's natural arms. The work will advance a fundamental understanding of (1) human interface performance in physical workspaces where cognition is embodied in both the human and supernumerary robot; (2) cognitive load and neuroplasticity using EEG and other physiological measurements; and (3) improvements in tri-manual coordination and manipulation capability arising from short-term training.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

超数机器人是一种辅助设备，可以附着在人体上，并被控制与人的自然肢体进行协调动作。冗余机器人有可能降低风险并提高人类在体力要求和/或危险工作场所活动中的能力，包括建筑，农业应用，搜索和救援以及外层空间探索。本计画的研究目标是在“第三手臂”超数机器人的即时灵巧控制的背景下，增进对人机互动的基本了解。该项目将开发一种新型的人机界面，允许人类使用肌电图活动（EMG）以闭环方式使用来自视觉和补充振动触觉信息的多模态感觉反馈来命令机器人。该项目团队将在两组人类受试者实验中使用新系统，旨在探索人类-机器人二元体中发生的双向适应，因为人类学会在新的三手操作任务中控制机器，即使机器人控制器适应肌肉EMG的变化信号特性。该项目推进了国家科学基金会的使命，即通过推进对共同适应的人机伙伴关系的基本理解，促进科学进步，促进国家健康，繁荣和福利。该项目的更广泛影响包括为小学和中学女生设立暑期课程，介绍机器人原理、传感器、神经科学和人类表现。该暑期项目与美国国家科学基金会的目标直接一致，即支持旨在提高妇女和其他代表性不足的群体在科学和技术领域的参与度的活动。该项目探讨了在“第三只手臂”多余机器人背景下的人机合作。计划进行两组人类受试者实验。 第一个建立了一个新的，自适应的，基于EMG的方法的可行性触觉引导的闭环控制的一个多余的机器人手臂的末端执行器。将记录对认知状态敏感的几种生物信号，包括瞳孔测量、皮肤电反应、心率变异性和脑电图（EEG）。受试者将执行点对点轨迹任务和3D版本的费茨定律任务，以评估在有和没有振动触觉性能反馈应用于用户足部的情况下的控制效率。第二个探讨了耦合动力学的短期人类学习的三手操作任务，需要用户协调他们的双手的运动与那些多余的机器人操纵一个新的三元素对象，以重建特定的所需的姿势。与此同时，机器人抓握力的在线计算学习适应将由用户自然手臂引起的物体运动学和动力学的扰动驱动。这项工作将推进对以下问题的基本理解：（1）在物理环境中的人机界面性能，其中认知体现在人类和多余机器人中;（2）使用EEG和其他生理测量的认知负荷和神经可塑性;及（3）改善三人手协调及操作能力，该奖项反映了NSF的法定使命，并被认为是值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估的支持。

项目成果

Classifiers and Adaptable Features Improve Myoelectric Command Accuracy in Trained Users

分类器和适应性功能提高了受过训练的用户的肌电命令准确性

• DOI: 10.1109/ner49283.2021.9441256
• 发表时间: 2021
• 期刊: 10th International IEEE/EMBS Conference on Neural Engineering (NER
• 作者: O'Meara, Sarah M.;Robinson, Stephen K.;Joshi, Sanjay S.
• 通讯作者: Joshi, Sanjay S.

Pilot Study for Myoelectric Control of a Supernumerary Robot During a Coordination Task

多余机器人协调任务期间肌电控制的初步研究

• DOI: 10.1007/978-3-031-05409-9_38
• 发表时间: 2022
• 期刊: Human-Computer Interaction. Technological Innovation. HCII 2022. Lecture Notes in Computer Science
• 作者: O’Meara, S.;Robinson, S.;Joshi, S.
• 通讯作者: Joshi, S.