Human-centered Robot Manipulation Planning for Solving Object Handover Tasks in the Real-World

旨在帮助人们日常生活的机器人需要具备强大的技能，能够在各种环境中将任意的未知物体交给他们的互动伙伴。老年人，特别是残疾人，往往难以操纵，甚至需要帮助做一些小家务，如移交电视遥控器，取水瓶和其他物品，如药品。从COVID-19爆发以来，这种援助需求变得更加明显，需要具有物体交接技能的协作机器人将受影响的人及其看护人保持在安全距离。同样，工厂中的这种机器人能力可以通过向协作工人移交各种工具来显着提高工作效率。然而，尽管具有物体移交技能的机器人具有重要意义，但这样一项基本任务仍然没有解决。该提案提出了一个框架，以解决在不受控制的环境中与任意日常生活对象的人机交接任务，并明确考虑了运动障碍患者（如肌萎缩侧索硬化症）最常用的项目。该提案的技术贡献分为三个研究方向。首先，一种新的任务感知，3D姿态预测方法将被引入到预测未来的姿态的整个人体和他们的手持物体从原始的感官信息。在推理过程中，对机器人决策和控制至关重要的各种人体部位也将通过基于学习的注意力模型突出显示。其次，该建议将形式化，表示和学习特定于任务的物理人-对象和对象-对象交互，以预测社会可行的目标对象姿势，用于在操作过程中考虑预期的人类行为和机器人的运动可达性。第三，预测的人类行为和期望的对象移交姿势将被用来确定人类友好的机器人把握和生成通知人类感知的机器人运动序列。最后，建议的研究重点将被集成到一个统一的框架，以解决人与机器人和机器人与人的交接任务与任意未知的对象从原始视觉观察。该提案的成果还将展示新的概念验证，在现实世界中使用运动障碍者最常用的各种物品进行人机交接演示。该项目得到了跨董事会机器人基础研究计划的支持，由工程局（ENG）和计算机与信息科学与工程局（CISE）共同管理和资助该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Robots aiming to assist people in their daily lives need to have robust skills in handing over arbitrary, unknown objects to and from their interacting partners in various environments. Older people, especially those with a disability, often have difficulty maneuvering and need assistance even for minor chores such as handing over the TV remote, fetching water bottles, and other items such as medicines. This need for assistance has become even clearer from the COVID-19 outbreak, requiring collaborative robots with object handover skills to keep the affected person and their caretakers at a safe distance. Similarly, such robot abilities in factories can significantly improve work efficiency by handing over various tools to and from their collaborating workers. However, despite the significance of having robots with object handover skills, such a fundamental task remains unsolved. This proposal presents a framework to solve the human-robot handover tasks with arbitrary daily-life objects in uncontrolled environments and explicitly considers the most-used items by patients with motor impairments such as Amyotrophic Lateral Sclerosis.The technical contributions of this proposal are divided into three research thrusts. First, a novel task-aware, 3D pose forecasting approach will be introduced to predict future poses of the full human body and their handheld objects from raw sensory information. During inference, various human body parts that are crucial for robot decision-making and control in solving human-robot object handover tasks will also be highlighted through learning-based attention models. Second, the proposal will formalize, represent, and learn the task-specific physical human-object and object-object interactions to predict socially feasible target object poses for handovers concerning the expected human behaviors and robot’s kinematic reachability during manipulation. Third, the predicted human behaviors and desired object handover poses will be used to determine human-friendly robot grasp and generate informed human-aware robot motion sequences. Finally, the proposed research thrusts will be integrated into a unified framework to solve human-to-robot and robot-to-human handover tasks with arbitrarily unknown objects from raw visual observations. The proposal’s outcomes will also exhibit new, proof-of-concept, human-robot handover demonstrations in the real-world using various most-used items by people with motor impairments.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.