CAREER: Robotic Manipulation Using Deep Deictic Reinforcement Learning

职业：使用深度指示强化学习的机器人操作

• 批准号: 1750649
• 负责人: Robert Platt
• 金额: $ 49.99万
• 依托单位: Northeastern University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1750649&HistoricalAwards=false
• 关键词: CAREER; Robotic; Manipulation; Using; Deep

As robot tasks and environments become more complex, it is getting too challenging to program every detail of the robot's behavior explicitly, by hand. An alternate approach is to learn behaviors through experience, a type of machine learning known as ``reinforcement learning'', where the robot learns through trial and error. Pure trial and error, however, is inefficient, which means it takes the robot a long time to learn. The goal of this project is to enable robots to focus attention on the parts of the environment that lead to effective learning and good generalization to new tasks. A result of this research is the ability of assistive robots in the home, such as an assistive wheelchair equipped with a robotic arm, to learn how to better help the infirm and people with disabilities.This project will develop a new approach to applying deep reinforcement learning (deep RL) to robotic manipulation problems by incorporating deictic representations. A deictic representation encodes state/action relative to a marker that the agent places in the environment. In this project, the marker is a 6-DOF reference frame, placed in a 3-D point cloud, or truncated signed distance function. The robot decides where to place the marker and how it should move relative to that marker by solving a Markov decision process using deep reinforcement learning. Preliminary results suggest that this new method can enable robots to learn control policies that solve complex manipulation problems without the need for precise geometric models of the objects being manipulated. While the method still estimates some elements of object pose implicitly, it does so in a way that generalizes well to novel objects and does not necessarily estimate full object pose unless required by the task.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着机器人的任务和环境变得越来越复杂，手动显式编程机器人行为的每个细节变得越来越具有挑战性。 另一种方法是通过经验来学习行为，这是一种被称为“强化学习”的机器学习，机器人通过试验和错误来学习。然而，纯粹的试错是低效的，这意味着机器人需要很长时间来学习。 该项目的目标是使机器人能够将注意力集中在环境的各个部分上，从而有效地学习和良好地概括新任务。 本研究的成果之一是，在家庭中使用的辅助机器人，例如配备了机械臂的辅助轮椅，能够学习如何更好地帮助体弱者和残疾人。本项目将开发一种新的方法，将深度强化学习（deep RL）应用于机器人操作问题，通过引入指示表示。指示表示相对于智能体在环境中放置的标记对状态/动作进行编码。在这个项目中，标记是一个6自由度参考框架，放置在三维点云中，或截断有符号距离函数。机器人通过使用深度强化学习解决马尔可夫决策过程来决定将标记放置在何处以及如何相对于该标记移动。初步结果表明，这种新方法可以使机器人学习控制策略，解决复杂的操作问题，而不需要精确的几何模型的对象被操纵。虽然该方法仍然估计物体的一些元素构成隐含，它这样做的方式，以及推广到新的对象，并不一定估计完整的对象构成，除非要求的task.This奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

项目成果

Pick and Place Without Geometric Object Models

无需几何对象模型即可拾取和放置

• DOI: 10.1109/icra.2018.8460553
• 发表时间: 2018
• 期刊: Proceedings of 2018 IEEE International Conference on Robotics and Automation (ICRA
• 作者: Gualtieri, Marcus;Pas, Andreas ten;Platt, Robert
• 通讯作者: Platt, Robert

Sample Efficient Grasp Learning Using Equivariant Models

• DOI: 10.15607/rss.2022.xviii.071
• 发表时间: 2022-02
• 期刊: ArXiv
• 作者: Xu Zhu;Dian Wang;Ondrej Biza;Guanang Su;R. Walters;Robert W. Platt

Equivariant Transporter Network

• DOI: 10.15607/rss.2022.xviii.007
• 发表时间: 2022-02
• 期刊: ArXiv
• 作者: Hao-zhe Huang;Dian Wang;R. Walters;Robert W. Platt

Leveraging Fully Observable Policies for Learning under Partial Observability

• DOI: 10.48550/arxiv.2211.01991
• 发表时间: 2022-11
• 作者: Hai V. Nguyen;Andrea Baisero;Dian Wang;Chris Amato;Robert W. Platt

Guest Editorial Open Discussion of Robot Grasping Benchmarks, Protocols, and Metrics

客座社论关于机器人抓取基准、协议和指标的公开讨论

• DOI: 10.1109/tase.2018.2871354
• 发表时间: 2018
• 期刊: IEEE Transactions on Automation Science and Engineering
• 影响因子: 5.6
• 作者: Mahler, Jeffrey;Platt, Rob;Rodriguez, Alberto;Ciocarlie, Matei;Dollar, Aaron;Detry, Renaud;Roa, Maximo A.;Yanco, Holly;Norton, Adam;Falco, Joe
• 通讯作者: Falco, Joe