CAREER: Improving Multi-Fingered Manipulation by Unifying Learning and Planning

对于机器人来说，要想在日常生活中自主地充当助手，在危险环境中充当人类的代理人，或者在忙碌的工厂和加工中心充当工人，它们必须能够流畅地抓住和操纵它们以前从未遇到过的物体。本计画探讨利用多指机械手进行抓取及手内操作之新方法。这项工作的主要新奇之处来自于研究新的方法相结合的知识自动获得的机器人从自己的传感器与模型给机器人的程序员。研究团队将对所开发的算法进行广泛的实证评估，以量化这些新技术相对于先前提出的分析或数据驱动方法的优势。这将使研究成果能够快速转化为工业和服务环境中使用的机器人。本项目将开展利用机器人操作的教育推广活动，以提高和加强中学生对计算的兴趣。为了使自主机器人能够在人类环境中无缝地作为助手进行操作，必须赋予它们适当的灵巧操作能力。当代机器人硬件提供了必要的灵活性和灵敏度来执行灵巧的操作，但算法的缺点目前削弱了部署强大的多指抓取，重新抓取，和手操纵未知的和部分建模的对象。该项目的研究目标是统一基于模型的规划和数据驱动的操作学习的概念，以提高对未知物体的灵巧操作。第一个研究重点是研究添加基于模型的约束，以使用学习的深度网络执行抓取合成。第二个研究重点评估的假设，学习反馈政策，近似模型，从触觉和视觉传感的图形结构将提高执行预先计划的手操纵轨迹。 研究人员建议将这些问题视为概率推理的一个实例。这使机器人能够直接推理不确定的感官观察和部分对象的姿态和接触状态的信息。这个奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

For robots to act autonomously as assistants in one's daily life, as surrogates for humans in dangerous environments, or as workers in busy factories and processing centers, they must be able to fluently grasp and manipulate objects that they have never previously encountered. This project investigates new approaches to perform grasping and in-hand manipulation using multi-fingered robot hands. The primary novelty in this work comes from examining new ways of combining knowledge gained automatically by the robot from its own sensors with models given to the robot by its programmer. The research team will perform extensive empirical evaluation of the algorithms developed in order to quantify the benefit of these new techniques over previously proposed analytic or data-driven approaches. This will allow for fast translation of the research results into robots used in industrial and service settings. This project will conduct educational outreach activities using robot manipulation to increase and reinforce middle and high schoolers' interest in computing.For autonomous robots to operate seamlessly as assistants in human environments, they must be endowed with the ability to aptly perform dexterous manipulation. Contemporary robot hardware provides the necessary dexterity and sensitivity to perform dexterous manipulation, but algorithmic shortcomings currently cripple deployment of robust multi-fingered grasping, regrasping, and in-hand manipulation of unknown and partially modeled objects. The research goal of this project is to unify concepts from model-based planning and data-driven learning for manipulation to improve dexterous manipulation of unknown objects. The first research thrust examines adding model-based constraints to perform grasp synthesis with a learned deep network. The second research thrust evaluates the hypothesis that learning feedback policies, approximate models, and graph structures from tactile and visual sensing will improve execution of pre-planned in-hand manipulation trajectories. The investigator proposes viewing these problems as an instance of probabilistic inference. This enables the robot to directly reason over uncertain sensory observations and partial object-pose and contact-state information.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.