SHF: Small: High-Performance Multi-Agent Reinforcement Learning

人工智能（AI）已迅速成为一个关键领域，应用于自动驾驶、机器人、航空航天、医疗保健等领域。对于一台机器（人工智能代理）来说，要想模仿人类的行为并有效地运行，它应该在环境中运行时同时具备强大的决策和学习能力。多智能体强化学习（MARL）是一个很有前途的研究领域，可以建模和控制多个分布式决策AI代理。然而，最近的研究表明，MARL算法效率低下，这可能会严重限制它们在现实世界系统中的采用。这些问题的发生是由于决策过程的复杂性，因为必须观察和处理环境中存在的大量事件，沿着需要相互交互的AI代理数量的增长。为了改善MARL算法的学习效率和可扩展性问题，项目研究人员采用了一种新的跨学科解决方案方法，利用计算机架构，机器学习理论和优化。具体来说，该项目将寻求技术来提高神经网络的吞吐量，以动态的方式有效地管理状态-动作空间，并可扩展地编码大量不同数量的代理的状态和观察。采用软硬件协同设计方法，加速软硬件层的并行优化。该项目的研究成果将显著提高MARL框架在现实世界中的应用，并对大学课程开发和计算行业产生积极影响。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Artificial Intelligence (AI) has rapidly become a critical domain with applications in autonomous driving, robotics, aerospace, healthcare, and others. For a machine (AI agent) to closely mimic human behavior and operate effectively, it should possess the capabilities of robust decision making and learning simultaneously as it operates in the environment. Multi-Agent Reinforcement Learning (MARL) is a promising research area that can model and control multiple distributed decision-making AI agents. However, recent studies have shown that the MARL algorithms suffer from inefficiencies that can severely limit their adoption in real-world systems. These problems occur due to complexities in decision-making processes arising from having to observe and act upon a large number of events present in the environment, along with the growth in the number of AI agents needed to interact with each other.To ameliorate the learning efficiency and scalability issues of MARL algorithms, the project investigators adopt a novel interdisciplinary solution approach, harnessing computer architecture, machine-learning theory and optimization. Specifically, the project will seek techniques to improve neural-network throughput, to efficiently manage the state-action space in a dynamic fashion and to scalably encode states and observations of a large and varying number of agents. A hardware-software co-design approach is adopted to accelerate the concurrent optimization of software and hardware layers. The research outcomes of this project will significantly enhance the adoption of MARL frameworks in real-world applications and positively impact university curricular development and the computing industry.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.