RI: Medium: Building Flexible, Robust, and Autonomous Agents

该项目正在开发计算代理，这些代理在丰富和动态的环境中长时间运行，并在没有特定任务编程的情况下掌握其环境的许多方面。为了实现这些目标，研究人员正在探索一个认知架构的空间，其中包含真实的神经回路的四个基本特征：（1）强化导致内在奖励的行为，（2）执行和学习心理以及运动动作，（3）提取心理表征中的特征，无论是来自感知还是认知操作，（4）持续编码和提取过去事件的情景记忆。一个名为Storm的软件框架通过集成独立的功能子系统来促进这种探索，允许研究人员轻松插入和删除不同的子系统，以评估它们对系统整体行为的影响。认知架构正在通过将其暴露于各种各样的具有不可预测（和不可重复）的外在奖励的新环境中来进行测试，但其中许多行为可能导致内在奖励（例如，惊喜）。为了评估灵活性，自动化环境生成器将代理暴露在人工代理或人类研究人员事先未知的环境中。为了评估鲁棒性，认知系统被暴露在同一环境的许多变体中，以确保系统可以从过去的经验中学习，并在适当的时候进行概括。为了评估自主性，系统必须在动态环境中有效地运行很长一段时间。从长远来看，在这项研究下开发的灵活和强大的认知架构将作为机器人和软件系统的“大脑”应用于紧急情况，军事和各种其他社会和服务领域。该奖项是根据2009年美国复苏和再投资法案（公法111-5）资助的。

This project is developing computational agents that operate for extended periods of time in rich and dynamic environments, and achieve mastery of many aspects of their environments without task-specific programming. To accomplish these goals, research is exploring a space of cognitive architectures that incorporate four fundamental features of real neural circuitry: (1) reinforcing behaviors that lead to intrinsic rewards, (2) executing and learning over mental, as well as, motor actions, (3) extracting regularities in mental representations, whether derived from perception or cognitive operations, and (4) continuously encoding and retrieving episodic memories of past events. A software framework called Storm facilitates this exploration by enabling the integration of independent functional subsystems, allowing researchers to easily plug in and remove different subsystems in order to assess their impact on the overall behavior of the system. Cognitive architectures are being tested by exposing them to a wide variety of novel environments with unpredictable (and non-repeatable) extrinsic rewards, but in which many actions could lead to intrinsic rewards (e.g., surprise). To assess flexibility, an automated environment generator exposes agents to environments that are unknown in advance to the artificial agent or human researcher. To assess robustness, cognitive systems are being exposed to many variants of the same environment to ensure that the systems can learn from past experience and generalize when appropriate. And to assess autonomy, systems' must operate effectively for extended periods of time in a dynamic environment. In the longer term, flexible and robust cognitive architectures being devloped under this research will have application as the 'brains' of robotic and software systems in emergency, miltary, and a wide variety of other societal and service realms. This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).