Coordination of Dyadic Object Handover for Human-Robot Interactions

人机交互的二元对象切换协调

• 批准号: 1935337
• 负责人: Eugene Tunik
• 金额: $ 76.03万
• 依托单位: Northeastern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1935337&HistoricalAwards=false
• 关键词: Coordination; Dyadic; Object; Handover; Human

The research objective of this project is to improve the fluidity of human-robot interactions wherein robots and humans can seamlessly pass objects between one another. Object handover is critical to everyday interactions, whether in ordinary environments or in high-stakes circumstances such as operating rooms. Seemingly effortless object handover results from successful inference and anticipation of shared intentions and actions. Manual object transfer between humans and robots will become increasingly important as robots become more common in the workplace and at home. The project team will perform human subject experiments investigating human-human and human-robot interactions within the context of object handover tasks to identify characteristics of dyadic coordination that allow people to understand their collaborator's intentions, to anticipate their actions, and to coordinate movements leading to task success. The team will use that new knowledge to develop robots that people can collaborate with on physical tasks as readily as they do other humans. Broader Impacts of the project include training opportunities for high school, undergraduate, and graduate students, with efforts to increase participation of underrepresented groups.This project explores human and robotic perception, behavior, and intent inference in a bidirectional and integrated manner within the context of object handover. Three specific aims are planned. The first uses motion capture, eye tracking, and electroencephalography data to build models of human intent and action during object handover. A novelty of the models is that they are sensitive to an individual's role (giver vs. receiver; leader vs. follower), to the presence or absence of communicative gaze, and to the degree of predictability of certain aspects of handover, such as grasp type, locus of handover, gaze conditions, and dyadic role. The second aim will develop a real-time intent inference engine that uses recursive Bayesian state estimation to obtain a probabilistic assessment of human intent as the handover action evolves in time. The output of this model will inform the robot's trajectory planner, thereby enabling it to make short time horizon predictions of human actions and to adjust robot motion plans accordingly in real-time. The third aim will examine how specific choices in the high-level planning and low-level control of robot motion impact human inference of robotic intent and action during object handover. If successful, this project will advance understanding of robot manipulation during human-robot handover and yield algorithms for achieving advanced autonomy during human collaboration with humanoid robots.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的研究目标是提高人-机器人交互的流畅性，其中机器人和人类可以在彼此之间无缝地传递对象。对象移交对于日常交互至关重要，无论是在普通环境中还是在高风险环境中，如手术室。看似毫不费力的物体移交源于对共同意图和行动的成功推理和预期。随着机器人在工作场所和家庭中变得越来越常见，人类和机器人之间的手动物体转移将变得越来越重要。项目团队将在物体移交任务的背景下进行人体实验，调查人与人和人与机器人的相互作用，以确定二元协调的特征，使人们能够理解他们的合作者的意图，预测他们的行动，并协调导致任务成功的动作。该团队将利用这一新知识来开发机器人，人们可以像与其他人类一样，在物理任务上轻松合作。该项目的更广泛影响包括为高中生、本科生和研究生提供培训机会，努力增加未被充分代表的群体的参与。该项目在对象移交的背景下以双向和集成的方式探索人类和机器人的感知、行为和意图推理。计划了三个具体目标。第一种方法使用动作捕捉、眼球跟踪和脑电数据来建立物体交接过程中人类意图和行为的模型。这些模型的一个新奇之处在于，它们对个体的角色(给予者与接受者；领导者与追随者)、交际性凝视的存在与否以及交接的某些方面的可预测性程度都很敏感，如握持类型、交接地点、凝视条件和二元角色。第二个目标将开发一个实时意图推理引擎，该引擎使用递归贝叶斯状态估计来获得随着移交动作的时间演变而对人类意图的概率评估。该模型的输出将通知机器人的轨迹规划器，从而使其能够对人类行动进行短期地平线预测，并相应地实时调整机器人的运动计划。第三个目标将研究在机器人运动的高层规划和低层控制中的具体选择如何影响人类对机器人意图和动作的推断。如果成功，这个项目将促进对人类-机器人交接过程中机器人操作的理解，并产生在人类与类人机器人合作期间实现高级自主性的算法。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Effects of Sensory Feedback and Collider Size on Reach-to-Grasp Coordination in Haptic-Free Virtual Reality

• DOI: 10.3389/frvir.2021.648529
• 发表时间: 2021-08-19
• 期刊: FRONTIERS IN VIRTUAL REALITY
• 作者: Furmanek, Mariusz P.;Mangalam, Madhur;Tunik, Eugene
• 通讯作者: Tunik, Eugene

Leveraging Submovements for Prediction and Trajectory Planning for Human-Robot Handover

利用子运动进行人机切换的预测和轨迹规划

• DOI: 10.1145/3529190.3529220
• 发表时间: 2022
• 期刊: Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments
• 作者: Lockwood, Kyle;Bicer, Yunus;Asghari-Esfeden, Sadjad;Zhu, Tianjie;Furmanek, Mariusz;Mangalam, Madhur;Strenge, Garrit;Imbiriba, Tales;Yarossi, Mathew;Padir, Taskin
• 通讯作者: Padir, Taskin

Motor cortex mapping using active gaussian processes

使用主动高斯过程进行运动皮层映射

• DOI: 10.1145/3389189.3389202
• 发表时间: 2020
• 期刊: PETRA
• 作者: Faghihpirayesh, Razieh;Imbiriba, Tales;Yarossi, Mathew;Tunik, Eugene;Brooks, Dana;Erdoğmuş, Deniz
• 通讯作者: Erdoğmuş, Deniz

An experimental and computational framework for modeling multi-muscle responses to transcranial magnetic stimulation of the human motor cortex.

用于模拟人类运动皮层经颅磁刺激的多肌肉反应的实验和计算框架。

• DOI: 10.1109/ner.2019.8717159
• 发表时间: 2019
• 期刊: International IEEE/EMBS Conference on Neural Engineering : [proceedings]. International IEEE EMBS Conference on Neural Engineering
• 作者: Yarossi,Mathew;Quivira,Fernando;Dannhauer,Moritz;Sommer,MarcA;Brooks,DanaH;Erdoğmuş,Deniz;Tunik,Eugene
• 通讯作者: Tunik,Eugene

Towards End-to-End Control of a Robot Prosthetic Hand via Reinforcement Learning

通过强化学习实现机器人假手的端到端控制

• DOI: 10.1109/biorob49111.2020.9224380
• 发表时间: 2020
• 期刊: 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob
• 作者: Sharif, Mohammadreza;Erdogmus, Deniz;Amato, Christopher;Padir, Taskin
• 通讯作者: Padir, Taskin