RI: Small: Goal-Driven Autonomy

目标驱动自治（Goal-driven autonomy，GDA）是一种反思性的目标推理模型，通过动态解决世界状态中的意外差异来控制智能体活动的焦点，这在复杂环境中解决任务时经常出现。这个项目的动机是两个观察GDA代理。首先，要表现良好，全面的GDA代理需要大量的领域知识，但是，很少有技术已经研究了学习这些知识。第二，虽然现有的GDA代理在各种任务中表现出良好的性能，理解和推广他们的成功已经阻碍了这些代理的目标模型和他们使用的表示之间的差距，例如，目前的大部分研究GDA代理假设代理的目标和行动的CNOPS表示。该项目旨在研究能够学习期望，解释和目标的GDA代理。该项目旨在开发能够创建GDA代理的方法，这些代理可以自主行动并学习：（1）识别他们所期望的与实际发生的情况之间存在差异的情况;（2）解释差异;（3）根据这些解释决定尝试实现哪些目标;以及（4）采取行动实现这些目标。在这项工作中，每个代理的目标是最大化其在强化学习中定义的预期回报。这种方法自然适合4步GDA循环，便于使用定义良好的强化学习框架研究关于GDA的属性，并且能够采用表示形式主义，例如随机策略（即，状态-动作对的概率分布），其自然地适合于表示GDA代理旨在与之交互的域中的GDA代理的动作。该项目旨在开发表征方法，结合联合收割机FOL（一阶逻辑）文字和行动的基础上，以概率来表示GDA elements.由于潜在的目标驱动的自治的大规模和广泛的应用，这项研究的潜在的广泛影响是显着的。随着自主计算设备和软件的普遍存在，对技术的需求越来越迫切，这种技术使系统能够识别它们对“世界”的期望，诊断它们，然后自我调整。这是计算机科学所有领域中普遍存在的问题。例如，在环境智能的一般领域，自动化系统，如空气质量控制系统，必须监视和控制各种设备;如果不是不可能的话，程序员很难预见这种系统将遇到的所有潜在情况。另一个例子是网络安全，考虑到当前网络的开放性以及新技术和服务的不断集成，提前对所有潜在威胁实施应对措施是不可行的;相反，基于代理的系统必须持续监控整个网络，学习和推理期望，并在遇到差异时自主采取行动。该项目包括一个积极的教育部分。具体来说，它计划（1）定期让本科生参与开发和测试项目的精心设计的组件;（2）创建一门关于自适应和自我意识的GDA代理的课程，超越代理，强化学习和规划课程的传统界限;以及（3）创建和传播GDA代理的测试平台，不仅包括项目的GDA代理，还包括模拟和代理-模拟接口。创建和传播测试平台将有助于弥补缺乏系统和代理模拟接口的问题，这一直是GDA代理教学的绊脚石。

Goal-driven autonomy (GDA) is a reflective model of reasoning about goals to control the focus of an agent's activities by dynamically resolving unexpected discrepancies in the world state, which frequently arise when solving tasks in complex environments. This project is motivated by two observations about GDA agents. First, to perform well, comprehensive GDA agents require substantial domain knowledge; however, few techniques have been investigated for learning this knowledge. Second, while existing GDA agents have demonstrated good performance in a variety of tasks, understanding and generalizing their successes has been hindered by a gap between the kinds of domains that these agents aim to model and the representations that they use; for instance, the bulk of current research on GDA agents assumes STRIPS representations of the agent's goals and actions. This project aims to study GDA agents that are capable of learning expectations, explanations, and goals. This project aims to develop methods that enable creation of GDA agents that can autonomously act and learn to: (1) identify situations where discrepancies take place between what they expect and what actually has happened; (2) explain the discrepancy; (3) decide which goals to try to achieve as a result of these explanations; and (4) act to accomplish these goals. In this work, the objective of each agent is to maximize its expected return as defined in reinforcement learning. This approach fits naturally with the 4-step GDA cycle, facilitates studying properties about GDA using the well-defined reinforcement learning framework, and enables the adoption of representation formalisms such as stochastic policies (i.e., probability distributions of state-action pairs), which are naturally suited to represent GDA agent's actions in the domains that GDA agents aim to interact with. This project aims to develop representational methods that combine FOL (First Order Logic) literals and actions with probabilities as the basis to represent GDA elements.The potential Broader Impact of this research is significant due to the potentially large and widespread applications of goal-driven autonomy. With the pervasive presence of autonomous computing devices and software, there is an increasingly pressing need for technology that enables systems to recognize discrepancies in what they expect from their 'worlds', diagnose them, and then adjust themselves. This is a ubiquitous problem in all areas of computer science. For example, in the general area of ambient intelligence, automated systems, such as an air quality control system, must monitor and control a variety of devices; it is very difficult, if not impossible, for a programmer to foresee all potential situations that such a system will encounter. Another example is cyber security where given the openness that characterizes current networks and the continuous integration of new technologies and services into them, it is not feasible to implement counter measures for all potential threats in advance; instead, an agent-based system must continuously monitor the overall network, learn and reason about expectations, and act autonomously when discrepancies are encountered. This project includes a vigorous educational component. Specifically, it plans to (1) regularly involve undergraduate students in developing and testing carefully scoped components of the project; (2) create a course on adaptive and self-aware GDA agents that transcends traditional boundaries in courses on agents, reinforcement learning, and planning; and (3) create and disseminate testbeds for GDA agents that include not only the project's GDA agents but also simulations and agent-simulation interfaces. Creating and disseminating testbeds will help remediate the lack of systems and agent-simulation interfaces that has been a repeated stumbling block for teaching about GDA agents.