Human-centered Robot Manipulation Planning for Solving Object Handover Tasks in the Real-World

旨在帮助人们日常生活的机器人需要具备强大的技能，能够在各种环境中将任意的、未知的物体交给或交给与它们互动的伙伴。老年人，尤其是那些有残疾的人，通常在行动上有困难，甚至在一些小事上也需要帮助，比如递电视遥控器、取水瓶和其他物品，比如药品。COVID-19疫情爆发后，这种援助需求变得更加明显，需要具有物体移交技能的协作机器人使受影响的人及其照顾者保持安全距离。同样，这种机器人在工厂中的能力可以通过将各种工具交给和交给合作工人来显着提高工作效率。然而，尽管拥有具有物体移交技能的机器人意义重大，但这一基本任务仍未得到解决。该方案提出了一个解决在非受控环境下具有任意日常生活对象的人机切换任务的框架，并明确考虑了肌萎缩性侧索硬化症等运动障碍患者最常用的物品。该提案的技术贡献分为三个研究重点。首先，将引入一种新的任务感知的3D姿势预测方法，根据原始感官信息预测整个人体及其手持物体的未来姿势。在推理过程中，还将通过基于学习的注意模型来突出在解决人机物体移交任务中对机器人决策和控制至关重要的人体各个部位。其次，该提案将形式化，表示和学习特定任务的物理人-对象和对象-对象交互，以预测社会可行的目标对象姿态，以便在操作过程中涉及预期的人类行为和机器人的运动学可达性。第三，预测的人类行为和期望的物体移交姿势将用于确定人类友好的机器人抓取并生成知情的人类感知机器人运动序列。最后，提出的研究重点将整合到一个统一的框架中，以解决具有原始视觉观察中任意未知物体的人对机器人和机器人对人的切换任务。该提案的成果还将在现实世界中展示新的、概念验证的人机交接演示，使用运动障碍患者最常用的各种物品。该项目由跨部门机器人基础研究项目支持，由工程（ENG）和计算机与信息科学与工程（CISE）联合管理和资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Robots aiming to assist people in their daily lives need to have robust skills in handing over arbitrary, unknown objects to and from their interacting partners in various environments. Older people, especially those with a disability, often have difficulty maneuvering and need assistance even for minor chores such as handing over the TV remote, fetching water bottles, and other items such as medicines. This need for assistance has become even clearer from the COVID-19 outbreak, requiring collaborative robots with object handover skills to keep the affected person and their caretakers at a safe distance. Similarly, such robot abilities in factories can significantly improve work efficiency by handing over various tools to and from their collaborating workers. However, despite the significance of having robots with object handover skills, such a fundamental task remains unsolved. This proposal presents a framework to solve the human-robot handover tasks with arbitrary daily-life objects in uncontrolled environments and explicitly considers the most-used items by patients with motor impairments such as Amyotrophic Lateral Sclerosis.The technical contributions of this proposal are divided into three research thrusts. First, a novel task-aware, 3D pose forecasting approach will be introduced to predict future poses of the full human body and their handheld objects from raw sensory information. During inference, various human body parts that are crucial for robot decision-making and control in solving human-robot object handover tasks will also be highlighted through learning-based attention models. Second, the proposal will formalize, represent, and learn the task-specific physical human-object and object-object interactions to predict socially feasible target object poses for handovers concerning the expected human behaviors and robot’s kinematic reachability during manipulation. Third, the predicted human behaviors and desired object handover poses will be used to determine human-friendly robot grasp and generate informed human-aware robot motion sequences. Finally, the proposed research thrusts will be integrated into a unified framework to solve human-to-robot and robot-to-human handover tasks with arbitrarily unknown objects from raw visual observations. The proposal’s outcomes will also exhibit new, proof-of-concept, human-robot handover demonstrations in the real-world using various most-used items by people with motor impairments.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.