Formal Framework for Analysis of Adaptation in Multi-Agent Systems (ADAPT2)

这项研究将为自适应多智能体系统集体行为的数学分析提供一个通用的框架。自适应是自主多智能体系统在不确定的动态环境中工作的基本要求，例如分布式机器人团队、嵌入式系统中的模块、传感器网络中的节点或软件智能体。适应使代理人能够改变他们的行为，以响应环境的变化或其他代理人的行动。对自适应系统的数学分析将使研究人员能够设计更健壮的系统，并预测、控制和了解它们的行为。这项研究将研究根据本地信息自主做出决策的代理，这些信息要么来自与其他代理的交互，要么来自本地环境。特别是，该项目将研究不同类别的适应行为，例如通过加强适应和通过空间扩展领域的交流进行适应。强化学习是一个强大的框架，在其中，代理通过对环境的试验和错误探索以及通过获得良好行动的奖励来学习最优行动。集体适应也可以发生在代理人通过外部场耦合的系统中，例如通过它们存放在环境中的标记。尽管适应和学习长期以来一直是人工智能界的焦点，但研究一组适应智能体将如何行动的工作相对较少。困难源于这样一个事实，即代理在其他适应性代理存在的情况下进行适应。通常情况下，事先不清楚该系统将如何运作，甚至不清楚适应是否会实现预期的目标。此外，设计者几乎没有指导，关于需要哪些个体代理特征来保证所需的集体行为。由于缺乏对这些问题的正式理解，研究人员无法充分利用这一强大的设计范例。这项研究将进行的数学分析将有助于回答这些问题。迫切需要更好的基础和工具来分析多智能体行为和验证多智能体系统的控制机制。缺乏这类工具阻碍了这类系统的更广泛部署，特别是机器人和嵌入式系统。验证控制算法所需的实验和仿真既耗时又昂贵。本项目将开发的数学模型所提供的定量理解将导致更稳健和更有效的控制算法，并在外地更多地部署这类系统。

This research will develop a general framework for mathematical analysis of collective behavior of adaptive multi-agent systems. Adaptation is an essential requirement for autonomous multi-agent systems functioning in uncertain dynamic environments, for example, distributed robot teams, modules in an embedded system, nodes in a sensor network, or software agents. Adaptation allows agents to change their behavior in response to changes in the environment or actions of other agents. Mathematical analysis of adaptive systems will enable researchers to design more robust systems, and to predict, control and understand their behavior. The research will study agents that make decisions autonomously based on local information, which comes either from interactions with other agents or from the local environment. In particular, this project will examine different classes of adaptive behavior, such as adaptation through reinforcement and adaptation through communication via spatially extended fields. Reinforcement learning is a powerful framework where an agent learns optimal actions through a trial and error exploration of the environment and by receiving rewards for good actions. Collective adaptation can also take place in systems in which agents are coupled through external fields, for example, through markers they deposit in the environment. Although adaptation and learning have long been the focus of the artificial intelligence community, there is relatively little work examining how a group of adaptive agents will act. The difficulty arises from the fact that agents adapt in the presence of other adaptive agents. Often it is not a priori clear how the system will act or even if adaptation will achieve the desired goals. In addition, the designer has very little guidance about what individual agent characteristics are required to guarantee the desired collective behavior. The lack of a formal understanding of these problems has prevented researchers from taking full advantage of this powerful design paradigm. The mathematical analysis to be performed in this research will help answer these questions. There is a critical need for better foundations and tools for analyzing multi-agent behavior and verifying control mechanisms for multi-agent systems. The lack of such tools stands in the way of wider deployment of such systems, especially robots and embedded systems. Experiments and simulations that are necessary to validate control algorithms are time consuming and costly. Quantitative understanding provided by the mathematical models to be developed in this project will lead to more robust and efficient control algorithms and greater deployment of such systems in the field.