SHF: Small: High-Performance Multi-Agent Reinforcement Learning

人工智能(AI)已迅速成为一个关键领域，应用于自动驾驶、机器人、航空航天、医疗保健等领域。为了使机器(人工智能代理)能够紧密模拟人类的行为并有效地运行，它应该具有在环境中运行时同时进行稳健决策和学习的能力。多智能体强化学习(MAIL)是一个可以对多个分布式决策AI智能体进行建模和控制的很有前途的研究领域。然而，最近的研究表明，Marl算法存在效率低下的问题，这严重限制了它们在现实系统中的采用。这些问题的出现是因为决策过程中的复杂性，因为必须观察环境中存在的大量事件并对其采取行动，以及相互交互所需的人工智能代理数量的增加。为了改善MAIL算法的学习效率和可扩展性问题，项目调查人员采用了一种新的跨学科解决方案方法，利用计算机体系结构、机器学习理论和优化。具体地说，该项目将寻求提高神经网络吞吐量的技术，以动态方式有效地管理状态-动作空间，并可伸缩地编码大量和不同数量的代理的状态和观察。采用软硬件协同设计的方法，加快了软硬件层的并行优化。该项目的研究成果将显著促进MAIL框架在现实世界应用中的采用，并对大学课程开发和计算行业产生积极影响。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Artificial Intelligence (AI) has rapidly become a critical domain with applications in autonomous driving, robotics, aerospace, healthcare, and others. For a machine (AI agent) to closely mimic human behavior and operate effectively, it should possess the capabilities of robust decision making and learning simultaneously as it operates in the environment. Multi-Agent Reinforcement Learning (MARL) is a promising research area that can model and control multiple distributed decision-making AI agents. However, recent studies have shown that the MARL algorithms suffer from inefficiencies that can severely limit their adoption in real-world systems. These problems occur due to complexities in decision-making processes arising from having to observe and act upon a large number of events present in the environment, along with the growth in the number of AI agents needed to interact with each other.To ameliorate the learning efficiency and scalability issues of MARL algorithms, the project investigators adopt a novel interdisciplinary solution approach, harnessing computer architecture, machine-learning theory and optimization. Specifically, the project will seek techniques to improve neural-network throughput, to efficiently manage the state-action space in a dynamic fashion and to scalably encode states and observations of a large and varying number of agents. A hardware-software co-design approach is adopted to accelerate the concurrent optimization of software and hardware layers. The research outcomes of this project will significantly enhance the adoption of MARL frameworks in real-world applications and positively impact university curricular development and the computing industry.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.