Human-centered Robot Manipulation Planning for Solving Object Handover Tasks in the Real-World

旨在帮助人们日常生活的机器人需要拥有强大的技能，能够在各种环境中向交互伙伴传递任意未知的对象。老年人，特别是残疾人，经常行动不便，甚至需要帮助，即使是一些琐碎的琐事，如交出电视遥控器，取水瓶，以及其他物品，如药品。这种对援助的需求从新冠肺炎疫情中变得更加明显，需要具有物体移交技能的协作机器人，以使受影响的人和他们的照顾者保持安全距离。同样，工厂中的这种机器人能力可以通过向协作工人传递各种工具以及从他们合作的工人那里传递各种工具来显著提高工作效率。然而，尽管拥有具有物体移交技能的机器人具有重要意义，但这样的基本任务仍然没有解决。该方案提出了一个解决非受控环境中任意日常生活物体的人-机器人交接任务的框架，并明确考虑了肌萎缩侧索硬化症等运动障碍患者最常用的物品。首先，将引入一种新的任务感知的3D姿势预测方法，以根据原始感觉信息预测整个人体及其手持对象的未来姿势。在推理过程中，对机器人决策和控制至关重要的各种人体部位也将通过基于学习的注意力模型得到突出显示。其次，该方案将形式化、表示和学习特定于任务的物理人-物和物-物相互作用，以预测关于预期的人类行为和机器人在操作过程中的运动学可达性的社会可行的目标对象姿势用于移交。第三，预测的人类行为和期望的物体交接姿势将被用来确定对人类友好的机器人抓取和生成知情的人类感知的机器人运动序列。最后，提出的研究推力将被集成到一个统一的框架中，以解决来自原始视觉观测的具有任意未知对象的人到机器人和机器人到人的切换任务。该提案的结果还将展示新的、概念验证的、人类-机器人交接演示，在现实世界中使用各种运动障碍人士最常用的物品。该项目由跨部门机器人基础研究计划支持，该计划由工程总监(ENG)和计算机和信息科学与工程(CEISE)共同管理和资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Robots aiming to assist people in their daily lives need to have robust skills in handing over arbitrary, unknown objects to and from their interacting partners in various environments. Older people, especially those with a disability, often have difficulty maneuvering and need assistance even for minor chores such as handing over the TV remote, fetching water bottles, and other items such as medicines. This need for assistance has become even clearer from the COVID-19 outbreak, requiring collaborative robots with object handover skills to keep the affected person and their caretakers at a safe distance. Similarly, such robot abilities in factories can significantly improve work efficiency by handing over various tools to and from their collaborating workers. However, despite the significance of having robots with object handover skills, such a fundamental task remains unsolved. This proposal presents a framework to solve the human-robot handover tasks with arbitrary daily-life objects in uncontrolled environments and explicitly considers the most-used items by patients with motor impairments such as Amyotrophic Lateral Sclerosis.The technical contributions of this proposal are divided into three research thrusts. First, a novel task-aware, 3D pose forecasting approach will be introduced to predict future poses of the full human body and their handheld objects from raw sensory information. During inference, various human body parts that are crucial for robot decision-making and control in solving human-robot object handover tasks will also be highlighted through learning-based attention models. Second, the proposal will formalize, represent, and learn the task-specific physical human-object and object-object interactions to predict socially feasible target object poses for handovers concerning the expected human behaviors and robot’s kinematic reachability during manipulation. Third, the predicted human behaviors and desired object handover poses will be used to determine human-friendly robot grasp and generate informed human-aware robot motion sequences. Finally, the proposed research thrusts will be integrated into a unified framework to solve human-to-robot and robot-to-human handover tasks with arbitrarily unknown objects from raw visual observations. The proposal’s outcomes will also exhibit new, proof-of-concept, human-robot handover demonstrations in the real-world using various most-used items by people with motor impairments.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.