RI: Small: Coordination in tightly coupled domains: Stepping stone rewards to induce the correct joint actions

点击翻译按钮获取中文摘要

This project introduces a new multiagent learning approach that leads to coordinated behavior in tightly coupled domains, that is, in domains where all agents must do the right thing at the right time for the team to achieve its goals. For example, getting a team of agents to lift and move an object heavier than the payload capacity of any single agent requires a sufficient number of agents to perform the correct action at the correct time. Unfortunately, most current learning methods fail in such situations because they rely on reinforcing the correct agent behavior only after the agents stumble upon the right actions. But what if the agents never jointly find the right actions? This project addresses this issue by introducing "stepping-stone rewards" that incentivize agents to perform the right actions even if their teammates have not yet found the correct complementary actions. The impact of this project will be to create larger and more capable multiagent teams that can be deployed in industry (such as factory robots that are not limited to a single task), in the field (such as autonomous search and rescue systems), in education (such as interactive learning via online gameplay) and in the home (such as networks of smart appliances).The main technical contribution of this project is to shift the learning problem faced by an agent from "did I take the correct action?" to "would my action have been correct had other agents taken the complementary action?" In tightly coupled multiagent domains, the first question results in very little positive feedback, creating a difficult to impossible learning problem. The new stepping stone rewards leverage hypothetical partners (partners that are surmised by an agent to explore the joint-action space) to overcome this difficulty by assessing the potential benefits of a particular action. Intuitively, stepping-stone rewards create a gradient for the agents to follow to enable fast and efficient learning in tightly coupled domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.