权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Career: IIS: RI: Improving Multi-Agent Reinforcement Learning for Cooperative, Partially Observable Settings

职业：IIS：RI：改进合作、部分可观察设置的多智能体强化学习

基本信息

批准号：
2044993
负责人：
Christopher Amato
金额：
$ 55万
依托单位：
Northeastern University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-03-01 至 2026-02-28
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2044993&HistoricalAwards=false
关键词：
Career IIS RI Improving Multi

项目摘要

As intelligent systems become more prevalent, these systems will need to coordinate with each other (e.g., apps, robots, autonomous cars), resulting in multi-agent systems. Allowing multi-agent systems to learn will let them operate in more complex and realistic scenarios by adapting their behavior to fit specific needs. Reinforcement learning is a promising form of trial-and-error learning that has the potential to drastically improve outcomes in many multi-agent domains (e.g., warehouses, delivery), but new methods are required for coordinating the agents in realistic domains with noisy and limited communication and sensing (i.e., partial observability). This project will develop these new reinforcement learning methods for coordinating teams of agents in various partially observable settings. The results will impact the development of future artificial intelligence (AI) and robotic systems and will be conveyed through outreach and educational activities.This project will develop a number of novel methods for cooperative multi-agent reinforcement learning (MARL) under partial observability. MARL, the extension of reinforcement learning methods for multi-agent domains, has gained popularity for generating high-quality solutions in some domains, but more work is needed to make the methods more scalable and widely applicable. Therefore, this project will first provide a better theoretical understanding of centralized training for decentralized execution methods. Centralized training for decentralized execution is the dominant paradigm in MARL where agents are trained offline and only executed online. The project will then develop new centralized training methods that are unbiased, scalable and perform well in a wide range of domains. Second, the project will develop online decentralized learning methods that allow agents to learn online even in noisy multi-agent settings. Lastly, to allow agents to learn and execute in an asynchronous manner, the project will develop methods for asynchronous MARL as well as asynchronous hierarchical learning with learning over multiple layers of a hierarchy. The resulting methods will significantly improve performance, stability and scalability of MARL methods and make them more generally applicable to large realistic domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着智能系统变得越来越普遍，这些系统将需要相互协调（例如，应用程序，机器人，自动汽车），从而产生了多智能体系统。允许多智能体系统学习将使它们能够通过调整其行为来适应特定需求，从而在更复杂和更现实的场景中运行。强化学习是试错学习的一种有前途的形式，它有可能极大地改善许多多智能体领域（例如，仓库、交付），但是需要新的方法来协调具有噪声和有限通信和感测的现实域中的代理（即，部分可观测性）。该项目将开发这些新的强化学习方法，用于在各种部分可观察的环境中协调代理团队。研究结果将影响未来人工智能（AI）和机器人系统的发展，并将通过推广和教育活动传达。本项目将开发一些新的方法，用于部分可观测性下的合作多智能体强化学习（MARL）。MARL是强化学习方法在多智能体领域的扩展，在某些领域中由于生成高质量的解决方案而受到欢迎，但需要做更多的工作才能使这些方法更具可扩展性和广泛适用性。因此，本项目将首先为分散执行方法的集中培训提供更好的理论理解。分散执行的集中式训练是MARL中的主导范式，其中代理人离线训练，仅在线执行。然后，该项目将开发新的集中培训方法，这些方法是公正的，可扩展的，并在广泛的领域中表现良好。其次，该项目将开发在线分散学习方法，允许代理即使在嘈杂的多代理设置中也能在线学习。最后，为了允许代理以异步方式学习和执行，该项目将开发异步MARL以及异步分层学习的方法，并在分层结构的多个层上进行学习。由此产生的方法将显着提高性能，稳定性和可扩展性的MARL方法，使他们更普遍地适用于大型现实domain.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。