CPS:Small: Collaborative Research: Distributed Coordination of Agents for Air Traffic Flow Management

本提案的目标是改善空中交通系统的管理，这是一个网络物理系统，在这个系统中，计算算法和物理系统之间的紧密连接对于安全、可靠和高效的性能至关重要。该方法基于自适应多智能体协调算法，特别强调智能体的系统选择，它们的行为和智能体的奖励函数。智力上的优点在于通过将焦点从“如何学习”转移到“学习什么”来解决物理环境中的代理协调问题。这种范式转换允许将代理使用的学习算法与用于将这些学习系统与系统性能联系起来的奖励函数分开。通过探索基于现实世界反馈隐式建模智能体交互的智能体奖励函数，这项工作旨在构建网络物理系统，其中智能体学习优化自己的奖励导致系统目标函数的优化。更广泛的影响是提供新的空中交通流量管理算法，这将大大减少空中交通拥堵。潜在的影响不仅可以用货币来衡量（2007年损失了410亿美元），还可以通过改善所有旅行者的体验来衡量，为社会带来重大利益。此外，pi将利用该项目培训研究生和本科生(i)通过开发交通系统多智能体学习的新课程；（ii）在NASA艾姆斯研究中心提供暑期实习机会。

CPS: Small: Collaborative Research: Distributed Coordination of Agents For Air Traffic Flow ManagementThis objective of this proposal is to improve the management of the air traffic system, a cyber-physical system where the need for a tight connection between the computational algorithms and the physical system is critical to safe, reliable and efficient performance. The approach is based on an adaptive multiagent coordination algorithm with a particular emphasis on the systematic selection of the agents, their actions and the agents' reward functions. The intellectual merit lies in addressing the agent coordination problem in a physical setting by shifting the focus from ``how to learn" to ``what to learn." This paradigm shift allows a separation between the learning algorithms used by agents, and the reward functions used to tie those learning systems into system performance. By exploring agent reward functions that implicitly model agent interactions based on feedback from the real world, this work aims to build cyber-physical systems where an agent that learns to optimize its own reward leads to the optimization of the system objective function. The broader impact is in providing new air traffic flow management algorithms that will significantly reduce air traffic congestion. The potential impact cannot only be measured in currency ($41B loss in 2007) but in terms of improved experience by all travelers, providing a significant benefit to society. In addition, the PIs will use this project to train graduate and undergraduate students (i) by developing new courses in multiagent learning for transportation systems; and (ii) by providing summer internship opportunities at NASA Ames Research Center.