RI: Small: Concept Formation in Partially Observable Domains

这项研究的重点是为人工智能(AI)系统提供表示问题领域知识的方法，方法是创建对该领域随时间变化的观察结果的描述。这项工作很重要，因为人工智能系统可以将他们的知识从一个问题领域转移到另一个问题领域，使它们能够随着时间的推移学习不同环境中的复杂行为。此外，学习的表示为创建对代理行为的解释提供了基础，随着人工智能代理被应用于我们日常生活的更多方面，这一能力正变得越来越重要。我们将创建的学习转移方法适用于更广泛的人工智能社区感兴趣的各种问题，包括可解释系统、智能可穿戴计算和现实世界环境中的机器人助手。通过这项工作实现的代理将自动从感知中提取概念(高级描述符)，构建经验层次结构，并在此结构上记录学习行为，通过这些新颖的表示扩展现有的强化学习方法。概念是简单、便携、高效的分层知识包，可以并行学习。我们的新贡献，基于概念的记忆，扩展了以前在概念形成方面的工作，以确定并非在所有上下文中都直接可观察到的域的有用属性，扩展了代理的世界模型，并提高了在部分可观察域中的性能。基于概念的记忆提供了用于创建域和该域中的任务的多层抽象表示的过程，使得能够跨多个任务进行学习迁移，并提供了创建对所学行为的解释的基础。我们在强化学习领域中的概念形成方法称为概念感知特征提取(CAFE)，它产生概念格表示，通过识别任务之间共同知识的适当泛化水平，允许从一个任务中学习的知识应用于新的问题。我们将通过将CAFE与抽象马尔可夫决策过程(AMDP)相集成来实现可扩展性，并通过开发启发式修剪方法来减少概念格的分支因素。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This research focuses on providing artificial intelligence (AI) systems with ways to represent knowledge about a problem domain, by creating descriptions of its observations over time. This work is important because the AI systems can transfer their knowledge from one problem domain to another, enabling them to learn complex behaviors in different environments over time. In addition, the learned representation provides a basis for creating explanations of the agent's behavior, a capability that is becoming increasingly important as AI agents are being applied to more aspects of our daily lives. The resulting learning transfer methods we will create are applicable to a wide variety of problems of interest to the broader AI community, including explainable systems, intelligent wearable computing, and robotic assistants in real world environments.The agents enabled by this work will automatically extract concepts (high-level descriptors) from perceptions, construct a hierarchy of experiences, and record learned behaviors over this structure, by extending existing reinforcement learning methods with these novel representations. Concepts serve as simple, portable, efficient packets of hierarchical knowledge that can be learned in parallel. Our novel contribution, concept-based memory, extends previous work on concept formation to identify useful properties of the domain that are not directly observable in all contexts, expanding the agent's world model and improving performance in partially observable domains. Concept-based memory provides a process for creating multi-layered abstract representations of a domain and the tasks in the domain, enabling learning transfer across multiple tasks, and providing a basis for creating explanations of learned behaviors. Our method for concept formation in reinforcement learning domains, called concept-aware feature extraction (CAFE), produces concept-lattice representations that permit knowledge learned from one task to be applied to a new problem by identifying the appropriate level of generalization for common knowledge between the tasks. We will enable scalability by integrating CAFE with abstract Markov decision processes (AMDPs) and by developing heuristic pruning methods that reduce the branching factor of the concept latticesThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.