权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

RI: Small: Coordination in tightly coupled domains: Stepping stone rewards to induce the correct joint actions

RI：小：紧密耦合领域中的协调：垫脚石奖励以诱导正确的联合行动

基本信息

批准号：
1815886
负责人：
Kagan Tumer
金额：
$ 40万
依托单位：
Oregon State University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-09-01 至 2023-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815886&HistoricalAwards=false
关键词：
RI Small Coordination tightly coupled

项目摘要

This project introduces a new multiagent learning approach that leads to coordinated behavior in tightly coupled domains, that is, in domains where all agents must do the right thing at the right time for the team to achieve its goals. For example, getting a team of agents to lift and move an object heavier than the payload capacity of any single agent requires a sufficient number of agents to perform the correct action at the correct time. Unfortunately, most current learning methods fail in such situations because they rely on reinforcing the correct agent behavior only after the agents stumble upon the right actions. But what if the agents never jointly find the right actions? This project addresses this issue by introducing "stepping-stone rewards" that incentivize agents to perform the right actions even if their teammates have not yet found the correct complementary actions. The impact of this project will be to create larger and more capable multiagent teams that can be deployed in industry (such as factory robots that are not limited to a single task), in the field (such as autonomous search and rescue systems), in education (such as interactive learning via online gameplay) and in the home (such as networks of smart appliances).The main technical contribution of this project is to shift the learning problem faced by an agent from "did I take the correct action?" to "would my action have been correct had other agents taken the complementary action?" In tightly coupled multiagent domains, the first question results in very little positive feedback, creating a difficult to impossible learning problem. The new stepping stone rewards leverage hypothetical partners (partners that are surmised by an agent to explore the joint-action space) to overcome this difficulty by assessing the potential benefits of a particular action. Intuitively, stepping-stone rewards create a gradient for the agents to follow to enable fast and efficient learning in tightly coupled domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这个项目引入了一种新的多代理学习方法，它可以在紧密耦合的域中导致协调行为，也就是在所有代理必须在正确的时间做正确的事情才能实现其目标的域中。例如，要让一组代理举起和移动比任何单个代理的有效载荷能力更重的对象，需要足够数量的代理才能在正确的时间执行正确的操作。不幸的是，大多数当前的学习方法在这种情况下都失败了，因为它们依赖于只有在代理偶然发现正确的操作之后才加强正确的代理行为。但如果代理人从未共同找到正确的行动怎么办？这个项目通过引入“踏脚石奖励”来解决这个问题，该奖励激励代理执行正确的操作，即使他们的队友还没有找到正确的补充操作。该项目的影响将是创建更大、更有能力的多智能体团队，这些团队可以部署在工业(如不限于单一任务的工厂机器人)、现场(如自主搜救系统)、教育(如通过在线游戏进行的互动学习)和家庭(如智能家电网络)。该项目的主要技术贡献是将智能体面临的学习问题从“我采取了正确的行动吗？”“如果其他特工采取补充行动，我的行动会正确吗？”在紧密耦合的多智能体领域中，第一个问题产生的正反馈非常少，从而产生了一个很难甚至不可能的学习问题。新的垫脚石奖励利用假设的合作伙伴(代理人猜测的探索联合行动空间的合作伙伴)通过评估特定行动的潜在好处来克服这一困难。直观地说，踏脚石奖励为代理创造了一个梯度，使他们能够在紧密耦合的领域中进行快速有效的学习。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。