RI: Small: Goal-Driven Autonomy

目标驱动的自主性(GDA)是一种关于目标的反射推理模型，通过动态地解决在复杂环境中解决任务时经常出现的世界状态中的意外差异来控制主体活动的焦点。这个项目的动机是关于GDA试剂的两个观察结果。首先，为了很好地执行，全面的GDA代理需要大量的领域知识；然而，很少有人研究过学习这些知识的技术。其次，虽然现有的GDA代理在各种任务中表现出了良好的性能，但这些代理旨在建模的域的类型与它们使用的表示之间的差距阻碍了对它们的成功的理解和概括；例如，当前对GDA代理的大部分研究假设代理的目标和操作的条带表示。这个项目旨在研究能够学习预期、解释和目标的GDA代理。该项目旨在开发能够创建能够自主行动的GDA代理的方法，并学习：(1)识别他们的预期与实际发生之间发生差异的情况；(2)解释差异；(3)根据这些解释决定试图实现哪些目标；以及(4)采取行动实现这些目标。在这项工作中，每个智能体的目标是最大化其在强化学习中定义的预期回报。这种方法自然地符合4步GDA循环，便于使用定义良好的强化学习框架来研究关于GDA的性质，并允许采用随机策略(即状态-动作对的概率分布)等表示形式，这些表示形式自然地适合于表示GDA代理在其目标交互的领域中的动作。该项目旨在开发将FOL(一阶逻辑)文字和动作与概率相结合来表示GDA元素的表征方法。由于目标驱动的自主性具有潜在的大规模和广泛的应用，因此本研究的潜在的更广泛的影响是显著的。随着自主计算设备和软件的普遍存在，人们对技术的需求日益迫切，这种技术使系统能够识别它们从他们的‘世界’中预期的差异，对它们进行诊断，然后进行自我调整。这是计算机科学各个领域普遍存在的问题。例如，在环境智能的一般领域，自动化系统，如空气质量控制系统，必须监测和控制各种设备；程序员很难预见这种系统将遇到的所有潜在情况，如果不是不可能的话。另一个例子是网络安全，鉴于当前网络的开放性以及新技术和服务不断融入其中，预先针对所有潜在威胁实施应对措施是不可行的；相反，基于代理的系统必须持续监控整个网络，了解并推理预期，并在遇到差异时自主采取行动。这个项目包括一个充满活力的教育部分。具体地说，它计划(1)定期让本科生参与开发和测试项目中仔细范围的组件；(2)创建一门关于适应性和自我意识的GDA代理的课程，它超越了代理、强化学习和规划课程中的传统界限；以及(3)为GDA代理创建和传播试验台，其中不仅包括项目的GDA代理，还包括模拟和代理-模拟接口。创建和传播试验台将有助于弥补缺乏系统和代理模拟接口，这一直是教授GDA代理的反复绊脚石。

Goal-driven autonomy (GDA) is a reflective model of reasoning about goals to control the focus of an agent's activities by dynamically resolving unexpected discrepancies in the world state, which frequently arise when solving tasks in complex environments. This project is motivated by two observations about GDA agents. First, to perform well, comprehensive GDA agents require substantial domain knowledge; however, few techniques have been investigated for learning this knowledge. Second, while existing GDA agents have demonstrated good performance in a variety of tasks, understanding and generalizing their successes has been hindered by a gap between the kinds of domains that these agents aim to model and the representations that they use; for instance, the bulk of current research on GDA agents assumes STRIPS representations of the agent's goals and actions. This project aims to study GDA agents that are capable of learning expectations, explanations, and goals. This project aims to develop methods that enable creation of GDA agents that can autonomously act and learn to: (1) identify situations where discrepancies take place between what they expect and what actually has happened; (2) explain the discrepancy; (3) decide which goals to try to achieve as a result of these explanations; and (4) act to accomplish these goals. In this work, the objective of each agent is to maximize its expected return as defined in reinforcement learning. This approach fits naturally with the 4-step GDA cycle, facilitates studying properties about GDA using the well-defined reinforcement learning framework, and enables the adoption of representation formalisms such as stochastic policies (i.e., probability distributions of state-action pairs), which are naturally suited to represent GDA agent's actions in the domains that GDA agents aim to interact with. This project aims to develop representational methods that combine FOL (First Order Logic) literals and actions with probabilities as the basis to represent GDA elements.The potential Broader Impact of this research is significant due to the potentially large and widespread applications of goal-driven autonomy. With the pervasive presence of autonomous computing devices and software, there is an increasingly pressing need for technology that enables systems to recognize discrepancies in what they expect from their 'worlds', diagnose them, and then adjust themselves. This is a ubiquitous problem in all areas of computer science. For example, in the general area of ambient intelligence, automated systems, such as an air quality control system, must monitor and control a variety of devices; it is very difficult, if not impossible, for a programmer to foresee all potential situations that such a system will encounter. Another example is cyber security where given the openness that characterizes current networks and the continuous integration of new technologies and services into them, it is not feasible to implement counter measures for all potential threats in advance; instead, an agent-based system must continuously monitor the overall network, learn and reason about expectations, and act autonomously when discrepancies are encountered. This project includes a vigorous educational component. Specifically, it plans to (1) regularly involve undergraduate students in developing and testing carefully scoped components of the project; (2) create a course on adaptive and self-aware GDA agents that transcends traditional boundaries in courses on agents, reinforcement learning, and planning; and (3) create and disseminate testbeds for GDA agents that include not only the project's GDA agents but also simulations and agent-simulation interfaces. Creating and disseminating testbeds will help remediate the lack of systems and agent-simulation interfaces that has been a repeated stumbling block for teaching about GDA agents.