RI: Small: Robot Developmental Learning of Skilled Actions

这个项目的目标是展示机器人-使用连续的视觉和触觉数据--如何学习在通常由人类完成的任务中以人类的技能水平工作。要在人的层面上发挥作用，它必须能够用诸如在盒子里放一个红色块这样的“对象级”抽象来规划，还必须能够抓住物体并移动它们，同时避免撞到物体并对周围造成损害。这个项目的灵感来自于人类的认知发展。婴儿通过从早期的规则和不可靠的动作到更复杂和可靠的动作的层次结构来学习对象和动作。要测试的假设是，这种自举学习方法允许机器人在人类主导的广泛环境中实现人类水平的熟练和健壮的动作。该项目借鉴了在基础知识表示和机器学习方面的大量先前工作。学习从检测低水平的偶然性开始-观察到的事件中的规律性-并将它们提炼成越来越准确的预测规则，可以用来定义可靠的行动。对于给定的规则，建立了一个简单的MDP模型，强化学习方法学习在动作层次的下一级完成动作的策略。习得的行动最初是不可靠的，但随着经验的积累，政策和行动会得到改善。注意力集中在学习可能最有成效的地方，通过内在动机方法奖励导致成功学习的行为，包括奖励模仿其他代理人成功行为的尝试这一重要的特殊情况。

The goal of this project is to show how a robot --- using a continuous stream of visual and tactile data --- can learn to work at a human level of skill in tasks normally done by humans. To function at a human level, it must be able to plan with "object-level" abstractions such as putting a red block into the box, and it must also be able to grasp objects and move them while avoiding bumping into things and causing damage to its surroundings. This project is inspired by human cognitive development. A baby learns about objects and actions by bootstrapping from early regularities and unreliable actions to hierarchies of more complex and reliable actions. The hypothesis to be tested is that this bootstrap learning approach allows a robot to achieve human levels of skillful and robust action in a wide range of human-dominated environments.This project draws on extensive prior work on foundational knowledge representations and machine learning. Learning begins by detecting low-level contingencies --- regularities among observed events --- and refining them into increasingly accurate predictive rules, that can be used to define reliable actions. For a given rule, a simple MDP model is formulated, and reinforcement learning methods learn a policy for accomplishing an action at the next level of the action hierarchy. Learned actions are initially unreliable, but policies and actions improve with experience. Attention is focused where learning is likely to be most productive by intrinsic motivation methods that reward actions that result in successful learning, including the important special case of rewarding attempts to imitate the successful actions of other agents.