CAREER: Robotic Manipulation Using Deep Deictic Reinforcement Learning

随着机器人的任务和环境变得越来越复杂，用手工明确地对机器人行为的每一个细节进行编程变得越来越具有挑战性。另一种方法是通过经验学习行为，这是一种被称为“强化学习”的机器学习，机器人通过反复试验来学习。然而，单纯的试错是效率低下的，这意味着机器人需要很长时间才能学习。该项目的目标是使机器人能够将注意力集中在环境中导致有效学习和对新任务的良好泛化的部分。这项研究的结果是家庭中的辅助机器人，如配备机械臂的辅助轮椅，学习如何更好地帮助体弱者和残疾人。本项目将开发一种新的方法，通过结合指示表征来应用深度强化学习(深度强化学习)来解决机器人操作问题。指示语表示相对于代理放置在环境中的标记对状态/动作进行编码。在本项目中，标记是放置在三维点云中的六自由度参考系，或截断符号距离函数。机器人通过使用深度强化学习求解马尔可夫决策过程来决定将标记放置在哪里以及它应该如何相对于该标记移动。初步结果表明，这种新方法可以使机器人学习解决复杂操作问题的控制策略，而不需要被操作对象的精确几何模型。虽然该方法仍然隐含地估计物体姿势的一些元素，但它以一种对新物体很好地概括的方式这样做，除非任务要求，否则不一定估计完整的物体姿势。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

As robot tasks and environments become more complex, it is getting too challenging to program every detail of the robot's behavior explicitly, by hand. An alternate approach is to learn behaviors through experience, a type of machine learning known as ``reinforcement learning'', where the robot learns through trial and error. Pure trial and error, however, is inefficient, which means it takes the robot a long time to learn. The goal of this project is to enable robots to focus attention on the parts of the environment that lead to effective learning and good generalization to new tasks. A result of this research is the ability of assistive robots in the home, such as an assistive wheelchair equipped with a robotic arm, to learn how to better help the infirm and people with disabilities.This project will develop a new approach to applying deep reinforcement learning (deep RL) to robotic manipulation problems by incorporating deictic representations. A deictic representation encodes state/action relative to a marker that the agent places in the environment. In this project, the marker is a 6-DOF reference frame, placed in a 3-D point cloud, or truncated signed distance function. The robot decides where to place the marker and how it should move relative to that marker by solving a Markov decision process using deep reinforcement learning. Preliminary results suggest that this new method can enable robots to learn control policies that solve complex manipulation problems without the need for precise geometric models of the objects being manipulated. While the method still estimates some elements of object pose implicitly, it does so in a way that generalizes well to novel objects and does not necessarily estimate full object pose unless required by the task.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.