S&AS: FND: COLLAB: Learning Manipulation Skills Using Deep Reinforcement Learning with Domain Transfer

这个项目开发了使用深度强化学习来解决现实世界机器人问题的新方法。该项目专注于机器人操作任务，如抓取、打开门、在家里帮忙、在海军舰艇上进行维修等。以上所有操作中的关键操作是机器人用手可靠地操作对象、部件或工具的能力，以执行任务。该项目利用深度强化学习：一种新的机器人学习方法，能够同时学习感知特征和控制策略。这个项目可能对各种实际应用有重要的好处，包括：我们军队的爆炸物处理，海军舰艇上的材料处理，NASA宇航员在太空中的灵巧机器人助手，可以帮助老年人更长时间在原地老化的辅助技术，更好地在核清理期间处理放射性材料的能力，帮助制造中符合人体工程学的任务，以及办公室和家庭的一般辅助。这项研究探索了新的深度强化学习方法，用于机器人抓取和操纵，在以前看不见的非结构化环境中工作得很好，并从更简单的子任务控制器组成端到端任务。这项研究是建立在研究团队最近工作的两个主要成果的基础上的，即掌握的深度学习方法和深度神经网络的领域适应方法。该研究遵循以下三个关键思想：1)在仿真中学习，然后使用域转移技术使解适应现实；2)通过规划估计值函数来简化视觉运动控制的学习；3)使用符号任务和运动规划来执行端到端的任务，通过对学习的控制器和规划的手臂/手运动进行排序。研究团队执行广泛的评估以确保系统能够执行任务的新实例，例如，在机器人以前未曾见过的环境中的那些实例。

This project develops new methods of using deep reinforcement learning to solve real world robotics problems. The project focuses on robotic manipulation tasks such as grasping, opening doors, helping out in the home, performing repairs aboard Navy ships, etc. The key operation in all of the above is the ability for the robot to reliably manipulate objects, parts, or tools with its hands in order to perform a task. The project leverages deep reinforcement learning: a new approach to robotic learning that is capable of learning both perceptual features and control policies simultaneously. This project could have important benefits for a variety of practical applications including: explosive ordnance disposal for our military, materials handling aboard Navy ships, dexterous robotic assistants for NASA astronauts in space, assistive technologies that could help seniors age in place longer, better capabilities for handling radioactive materials during nuclear cleanup, assistance for ergonomically challenging tasks in manufacturing, and general assistance in the office and the home.This research investigates novel deep reinforcement learning approaches for robotic grasping and manipulation that work well in previously unseen, unstructured environments and compose end-to-end tasks from simpler sub-task controllers. The research is built on two main results from research team's recent work, the deep learning approach to grasping and domain adaptation methods for deep neural networks. The research is guided by the following three key ideas: 1) learning in simulation and then using domain transfer techniques to adapt the solutions to reality; 2) simplifying learning for visuomotor control by using planning to estimate the value function; and 3) using symbolic task and motion planning to perform end-to-end tasks by sequencing learned controllers and planned arm/hand motions. The research team performs extensive evaluations to ensure that the system is able to perform novel instances of a task, e.g., those in a context that the robot has not seen before.