Robotic-Specific Machine Learning

该项目将开发特定于机器人的机器学习方法，使机器人能够有效地学习复杂的行为。对这种方法的要求直接来自没有免费午餐的定理(Wolpert，1996)，该定理证明，当对所有可能的问题进行平均时，没有任何机器学习方法比随机猜测更有效。改善随机猜测的唯一方法是限制问题空间，并将关于该问题空间的先验知识纳入学习方法。当然，有一些机器学习方法可以通过结合相当一般的先验知识来应用于广泛的现实世界任务，例如流畅性。然而，即使是解决相对简单的问题，这种方法也已经需要大量的数据和计算。机器人学的整体问题-学习行为将来自稀疏反馈的高维感觉输入流映射到高维马达输出流-太复杂了，无法通过使用现实数量的数据和计算的通用机器学习方法来解决。其他方法通过结合特定于任务的先验知识来避免该问题，例如通过为手头的机器人任务量身定做的工程特征和表示。然而，这些方法并不适用于新的任务。这个项目提出了一种介于一般方法和任务特定方法之间的学习机器人行为的中间基础。其关键思想是纳入特定于机器人的先验知识，即与广泛的机器人任务一致的先验知识。应用这一想法需要两个步骤：a)发现特定于机器人的先验知识，b)将这些先验知识纳入机器学习方法。我们可以通过观察机器人与物理世界相互作用的内在结构(例如，物理、体现、客观性)来发现这样的先验。为了结合这样的先验，我们将它们与内部状态表示相关联，这是从机器人的感觉输入到其电机输出的映射的中间结果。从技术上讲，我们将通过i)定义适当的学习目标和ii)限制假设空间来纳入机器人特定的先验知识。我们的工作将消除特定于任务的特征工程的需要，同时将数据和计算要求保持在最低限度。因此，这个项目将使机器人能够从原始的感官输入中自主学习复杂的任务。我们已经做了大量的前期工作，显示了我们方法的可行性和潜力。该项目将通过以下方式进一步发展这一想法：1)交互过程中的在线学习2)通过同时学习特定于任务的状态表示和递归循环来估计部分可观测任务来解决部分可观测任务3)学习结构化状态表示，使强化学习对于机器人任务更有效4)增量地学习多个相关任务的状态表示

This project will develop robotics-specific machine learning methods that enable robots to efficiently learn complex behavior. The requirement for such methods follows directly from the no-free-lunch theorems (Wolpert, 1996) which prove that no machine learning method works better than random guessing when averaged over all possible problems. The only way to improve over random guessing is to restrict the problem space and incorporate prior knowledge about this problem space into the learning method.Of course, there are machine learning methods that apply to a wide range of real world tasks by incorporating fairly general priors, e.g. smoothness. However, even for solving relatively simple problems, such methods already require huge amounts of data and computation. The overall problem of robotics---learning behavior that maps a stream of high-dimensional sensory input to a stream of high-dimensional motor output from sparse feedback---is too complex to be solved by generic machine learning methods using realistic amounts of data and computation. Other approaches avoid this problem by incorporating task-specific prior knowledge, e.g. by engineering features and representations tailored to the robotic task at hand. However, these approaches do not generalize to new tasks.This project proposes a middle ground between general and task-specific approaches to learning robot behavior. The key idea is to incorporate robotics-specific prior knowledge, i.e. priors that are consistent with a wide range of robotic tasks. Applying this idea requires two steps: a) discovering robotics-specific prior knowledge and b) incorporating these priors into machine learning methods. We can discover such priors by looking at the structure inherent in the interactions of robots and the physical world (e.g. physics, embodiment, objectness). To incorporate such priors, we will relate them to internal state representations, which are an intermediate result in the mapping from the robot's sensory input to its motor output. Technically, we will incorporate robotics-specific priors by i) defining appropriate learning objectives and by ii) restricting the hypothesis space. Our work will eliminate the need for task-specific feature engineering while keeping the data and computation requirements at a minimum. As a consequence, this project will enable robots to autonomously learn complex tasks from raw sensory input. We have extensive preliminary work showing the feasibility and the potential of our approach. This project will develop this idea further by:1) Online learning during the interaction2) Solving partially observable tasks by simultaneously learning task-specific state representations and recursive loops to estimate them3) Learning structured state representations that make reinforcement learning more efficient for robotic tasks4) Incrementally learning state representations for multiple related tasks