RI: Small: Robot Developmental Learning of Skilled Actions

这个项目的目标是展示一个机器人如何使用连续的视觉和触觉数据流来学习在通常由人类完成的任务中以人类的技能水平工作。为了在人类层面上发挥作用，它必须能够规划“对象级”抽象，例如将一个红色块放入盒子中，它还必须能够抓住对象并移动它们，同时避免撞到东西并对周围环境造成损害。这个项目的灵感来自于人类的认知发展。婴儿通过从早期的规律和不可靠的动作到更复杂和可靠的动作的层次结构来学习物体和动作。要测试的假设是，这种自举学习方法允许机器人在广泛的人类主导环境中达到人类水平的熟练和健壮的动作。该项目借鉴了之前在基础知识表示和机器学习方面的大量工作。学习开始于检测低水平的偶然性——观察到的事件之间的规律——并将它们提炼成越来越精确的预测规则，这些规则可以用来定义可靠的行动。对于给定的规则，制定了一个简单的MDP模型，强化学习方法学习在动作层次结构的下一层完成动作的策略。习得的行动最初是不可靠的，但政策和行动会随着经验而改进。注意力集中在学习最可能产生成效的地方，通过奖励导致成功学习的行为的内在动机方法，包括奖励模仿其他主体成功行为的重要特殊情况。

The goal of this project is to show how a robot --- using a continuous stream of visual and tactile data --- can learn to work at a human level of skill in tasks normally done by humans. To function at a human level, it must be able to plan with "object-level" abstractions such as putting a red block into the box, and it must also be able to grasp objects and move them while avoiding bumping into things and causing damage to its surroundings. This project is inspired by human cognitive development. A baby learns about objects and actions by bootstrapping from early regularities and unreliable actions to hierarchies of more complex and reliable actions. The hypothesis to be tested is that this bootstrap learning approach allows a robot to achieve human levels of skillful and robust action in a wide range of human-dominated environments.This project draws on extensive prior work on foundational knowledge representations and machine learning. Learning begins by detecting low-level contingencies --- regularities among observed events --- and refining them into increasingly accurate predictive rules, that can be used to define reliable actions. For a given rule, a simple MDP model is formulated, and reinforcement learning methods learn a policy for accomplishing an action at the next level of the action hierarchy. Learned actions are initially unreliable, but policies and actions improve with experience. Attention is focused where learning is likely to be most productive by intrinsic motivation methods that reward actions that result in successful learning, including the important special case of rewarding attempts to imitate the successful actions of other agents.