S&AS: FND: COLLAB: Learning Manipulation Skills Using Deep Reinforcement Learning with Domain Transfer

该项目开发了使用深度强化学习解决真实的世界机器人问题的新方法。该项目的重点是机器人操作任务，如抓取，开门，在家里帮忙，在海军舰艇上进行维修等，上述所有操作的关键是机器人能够可靠地操纵物体，零件或工具，以执行任务。该项目利用深度强化学习：一种新的机器人学习方法，能够同时学习感知特征和控制策略。该项目可能对各种实际应用产生重要的好处，包括：为我们的军队处理爆炸物，在海军舰艇上处理材料，为NASA宇航员在太空中提供灵巧的机器人助手，可以帮助老年人更长时间在原地老化的辅助技术，在核清理过程中处理放射性材料的更好能力，在制造业中协助人体工程学挑战性任务，这项研究调查了用于机器人抓取和操纵的新型深度强化学习方法，这些方法在以前看不见的非结构化环境中工作良好，并从更简单的子任务控制器组成端到端任务。该研究建立在研究团队最近工作的两个主要成果之上，即用于抓取的深度学习方法和用于深度神经网络的域自适应方法。该研究由以下三个关键思想指导：1）在模拟中学习，然后使用域转移技术来适应现实的解决方案; 2）通过使用规划来估计值函数，简化视觉控制的学习; 3）使用符号任务和运动规划来执行端到端的任务，通过排序学习的控制器和规划的手臂/手部运动。研究团队进行了广泛的评估，以确保系统能够执行任务的新实例，例如，那些机器人从未见过的场景。

This project develops new methods of using deep reinforcement learning to solve real world robotics problems. The project focuses on robotic manipulation tasks such as grasping, opening doors, helping out in the home, performing repairs aboard Navy ships, etc. The key operation in all of the above is the ability for the robot to reliably manipulate objects, parts, or tools with its hands in order to perform a task. The project leverages deep reinforcement learning: a new approach to robotic learning that is capable of learning both perceptual features and control policies simultaneously. This project could have important benefits for a variety of practical applications including: explosive ordnance disposal for our military, materials handling aboard Navy ships, dexterous robotic assistants for NASA astronauts in space, assistive technologies that could help seniors age in place longer, better capabilities for handling radioactive materials during nuclear cleanup, assistance for ergonomically challenging tasks in manufacturing, and general assistance in the office and the home.This research investigates novel deep reinforcement learning approaches for robotic grasping and manipulation that work well in previously unseen, unstructured environments and compose end-to-end tasks from simpler sub-task controllers. The research is built on two main results from research team's recent work, the deep learning approach to grasping and domain adaptation methods for deep neural networks. The research is guided by the following three key ideas: 1) learning in simulation and then using domain transfer techniques to adapt the solutions to reality; 2) simplifying learning for visuomotor control by using planning to estimate the value function; and 3) using symbolic task and motion planning to perform end-to-end tasks by sequencing learned controllers and planned arm/hand motions. The research team performs extensive evaluations to ensure that the system is able to perform novel instances of a task, e.g., those in a context that the robot has not seen before.