RI: Medium: Building Flexible, Robust, and Autonomous Agents

该项目正在开发在丰富和动态的环境中长时间运行的计算代理，并在没有特定任务编程的情况下实现对环境的许多方面的掌握。为了实现这些目标，研究人员正在探索一种认知结构空间，其中包含了真实神经电路的四个基本特征：(1)强化导致内在奖励的行为，(2)通过心理和运动动作来执行和学习，(3)提取心理表征中的规则，无论是来自感知还是认知操作，以及(4)连续编码和提取过去事件的情节记忆。一个名为Storm的软件框架通过支持独立功能子系统的集成促进了这种探索，使研究人员能够轻松地插入和移除不同的子系统，以便评估它们对系统整体行为的影响。认知架构正在接受测试，方法是将它们暴露在各种各样的新颖环境中，这些环境具有不可预测的(和不可重复的)外部奖励，但在这些环境中，许多行为可能会导致内在奖励(例如，惊喜)。为了评估灵活性，自动环境生成器将代理暴露在人工代理或人类研究人员事先未知的环境中。为了评估稳健性，认知系统暴露在同一环境的许多变体中，以确保系统能够从过去的经验中学习并在适当的时候进行推广。要评估自主性，系统必须在动态环境中长时间有效运行。从长远来看，在这项研究下开发的灵活而健壮的认知体系结构将作为机器人和软件系统在紧急情况、军事和其他各种社会和服务领域的“大脑”得到应用。该奖项是根据2009年《美国复苏和再投资法案》(公法111-5)提供资金的。

This project is developing computational agents that operate for extended periods of time in rich and dynamic environments, and achieve mastery of many aspects of their environments without task-specific programming. To accomplish these goals, research is exploring a space of cognitive architectures that incorporate four fundamental features of real neural circuitry: (1) reinforcing behaviors that lead to intrinsic rewards, (2) executing and learning over mental, as well as, motor actions, (3) extracting regularities in mental representations, whether derived from perception or cognitive operations, and (4) continuously encoding and retrieving episodic memories of past events. A software framework called Storm facilitates this exploration by enabling the integration of independent functional subsystems, allowing researchers to easily plug in and remove different subsystems in order to assess their impact on the overall behavior of the system. Cognitive architectures are being tested by exposing them to a wide variety of novel environments with unpredictable (and non-repeatable) extrinsic rewards, but in which many actions could lead to intrinsic rewards (e.g., surprise). To assess flexibility, an automated environment generator exposes agents to environments that are unknown in advance to the artificial agent or human researcher. To assess robustness, cognitive systems are being exposed to many variants of the same environment to ensure that the systems can learn from past experience and generalize when appropriate. And to assess autonomy, systems' must operate effectively for extended periods of time in a dynamic environment. In the longer term, flexible and robust cognitive architectures being devloped under this research will have application as the 'brains' of robotic and software systems in emergency, miltary, and a wide variety of other societal and service realms. This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).