EAGER: Memory-based learning of effective actions

这个项目解决了稳健智能中的基本问题，即自主代理如何学习使用低级别的亚符号(像素级)感官运动经验来学习更高级别的有效概念，从学习使用手操作桌面上的对象，到学习平衡和行走，再到学习在复杂的环境中移动而不与墙壁或行人碰撞。该项目将开发这种学习过程如何发生的计算模型，并将在实际的机器人上实施和测试这些计算模型。理解这种自主的概念学习有可能影响一系列学科，包括认知科学、心理学、人工智能，特别是机器人学、计算机视觉和机器学习。了解概念是如何在机器人导航的特定领域中形成和演变的，也有可能有助于帮助身体和学习障碍者的系统的进步。该项目借鉴了PI实验室两种不同方法的见解，这两种方法具有互补的优势：(1)行动和感知的定性学习者(QLAP)，和(2)模型预测平衡点控制(MPEPC)系统。QLAP系统利用对连续传感器输入的定性抽象来学习因果意外事件、DBN(动态信念网络)和因果世界的MDP模型，并建立动作模型的层次结构。它使用激光测距仪的感知和运动向量变化与感知向量中事件之间的相关峰值--所谓的偶然性--来识别产生的感知事件的运动信号，这些事件可能不仅仅是随机变化。可靠的情节可以作为案例被记住，并用于学习。MPEPC系统将移动机器人的连续导航问题分解为局部无约束控制和全局优化过程，以平衡进度和碰撞避免等约束。这两种方法都有一个局部阶段(学习偶发事件和局部控制律)和一个全局阶段(学习行动的层次结构并找到平衡约束的扩展路径)。这两种方法将通过从基于案例的推理(CBR)中学习的方法来增强，该方法利用呈现案例的特征从案例记忆中检索相关案例。将采用两个级别的案例陈述。最低级别的情况表示是一个简单的特征向量：在局部运动控制的情况下，它指定以自我为中心的参照系中的目标姿势位置，以及试图到达它的运动控制律的参数，以及结果轨迹的质量。将使用最近邻进行检索，通过局部加权回归或局部加权投影回归结合检索病例的信息。在更高层次的行动学习中，要通过确定决定行动全球结构的关键环境制约因素来描述案例。

This project addresses the foundational question in Robust Intelligence of how an autonomous agent can learn use low-level sub-symbolic (pixel-level) sensorimotor experiences with its environment to learn higher level effective concepts, ranging from learning to use a hand to manipulate objects on a tabletop, to learning to balance and walk, to learning to move through a complex environment without collisions with walls or pedestrians. This project will develop computational models of how this learning process could take place and will implement and test these computational models on an actual robot. Understanding such autonomous concept learning has the potential to impact a range of disciplines including Cognitive Science, Psychology, AI in general, and robotics, computer vision, and machine learning in particular. Understanding how concepts come into being and evolve in the specific domain of robot navigation also has the potential to contribute to advances in systems that help persons with physical and learning disabilities.The project draws on insights from two different approaches from the PI's lab that have complementary strengths: (1) QLAP (Qualitative Learner of Action and Perception), and (2) MPEPC system (Model Predictive Equilibrium Point Control). The QLAP system exploits a qualitative abstraction of continuous sensor input in order to learn causal contingencies, DBN (Dynamic Belief Network) and MDP models of the causal world, and to build a hierarchy of action models. It uses perception with laser rangefinders and correlation peaks between changes to the motor vector and events in the sense vector -- so-called contingencies -- to discern motor signals that produce resulting perceptual events that may be more than random variation. Reliable episodes can be remembered as cases and used in learning. The MPEPC system factors the continuous navigation problem for a mobile robot into a local unconstrained control and a global optimization process that balances constraints such as progress and collision avoidance. Both methods have a local phase (learning contingencies and local control laws), and a global phase (learning a hierarchy of actions and finding extended routes that balance constraints). These two approaches will be augmented by learning methods from Case-Based Reasoning (CBR) that use features of the presenting case to retrieve related cases from case memory. Two levels of case representation will be employed. The lowest level case representation is a simple feature vector: in the case of local motion control, it specifies the target pose location in the egocentric frame of reference, along with the parameters of the motion control law that attempts to reach it, and the quality of the resulting trajectory. Retrieval will be done using Nearest Neighbor, combining information from the retrieved cases by Locally Weighted Regression or Locally Weighted Projection Regression. At the higher level of action learning, a case is to be described by identifying the critical environmental constraints that determine the global structure of the action.