CAREER: Improving Multi-Fingered Manipulation by Unifying Learning and Planning

机器人要想在日常生活中充当助手，在危险环境中充当人类的替身，或者在繁忙的工厂和加工中心充当工人，它们必须能够熟练地掌握和操纵它们以前从未遇到过的物体。该项目研究了使用多指机器人手进行抓取和手部操作的新方法。这项工作的主要新颖之处在于研究了将机器人从自己的传感器自动获得的知识与程序员提供给机器人的模型相结合的新方法。研究团队将对所开发的算法进行广泛的实证评估，以量化这些新技术相对于先前提出的分析或数据驱动方法的好处。这将允许将研究成果快速转化为工业和服务环境中使用的机器人。该项目将利用机器人操作开展教育外展活动，以增加和加强初高中学生对计算机的兴趣。为了让自主机器人在人类环境中作为助手无缝地工作，它们必须具备适当地进行灵巧操作的能力。当代机器人硬件提供了执行灵巧操作所需的灵活性和灵敏度，但算法缺陷目前削弱了多指抓取、再抓取和手持操作未知和部分建模对象的部署。本课题的研究目标是将基于模型的规划和数据驱动的操作学习的概念统一起来，以提高对未知物体的灵巧操作。第一个研究重点是研究添加基于模型的约束来执行与学习深度网络的抓取综合。第二个研究重点评估了一个假设，即从触觉和视觉感知中学习反馈策略、近似模型和图形结构将改善预先计划的手持操作轨迹的执行。研究者建议将这些问题视为概率推理的一个实例。这使机器人能够直接推理不确定的感官观察和部分物体姿态和接触状态信息。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

For robots to act autonomously as assistants in one's daily life, as surrogates for humans in dangerous environments, or as workers in busy factories and processing centers, they must be able to fluently grasp and manipulate objects that they have never previously encountered. This project investigates new approaches to perform grasping and in-hand manipulation using multi-fingered robot hands. The primary novelty in this work comes from examining new ways of combining knowledge gained automatically by the robot from its own sensors with models given to the robot by its programmer. The research team will perform extensive empirical evaluation of the algorithms developed in order to quantify the benefit of these new techniques over previously proposed analytic or data-driven approaches. This will allow for fast translation of the research results into robots used in industrial and service settings. This project will conduct educational outreach activities using robot manipulation to increase and reinforce middle and high schoolers' interest in computing.For autonomous robots to operate seamlessly as assistants in human environments, they must be endowed with the ability to aptly perform dexterous manipulation. Contemporary robot hardware provides the necessary dexterity and sensitivity to perform dexterous manipulation, but algorithmic shortcomings currently cripple deployment of robust multi-fingered grasping, regrasping, and in-hand manipulation of unknown and partially modeled objects. The research goal of this project is to unify concepts from model-based planning and data-driven learning for manipulation to improve dexterous manipulation of unknown objects. The first research thrust examines adding model-based constraints to perform grasp synthesis with a learned deep network. The second research thrust evaluates the hypothesis that learning feedback policies, approximate models, and graph structures from tactile and visual sensing will improve execution of pre-planned in-hand manipulation trajectories. The investigator proposes viewing these problems as an instance of probabilistic inference. This enables the robot to directly reason over uncertain sensory observations and partial object-pose and contact-state information.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.