SELF-ORGANIZING MULTI-AGENT SYSTEMS : ARTIFICIAL LIFE APPROACHES

在试图让人工有机体或简单的反应式机器人合成一些协调行为的过程中，人工生命和机器人学领域的几名研究人员已经将单一的强化学习算法应用于多智能体学习问题。在大多数这些应用中，只有少数学习智能体参与它们的联合任务，因此每个智能体的状态空间相对较小。这就是为什么单片强化学习算法被成功地应用于这些多智能体学习问题的原因。然而，这些简单的强化学习算法的应用并不能成功地扩展到更复杂的多智能体学习问题，其中相当多的学习智能体参与一些协调任务。在这样的多智能体问题域中，智能体不仅要根据物理环境本身产生的感知信息行事，而且要根据其他智能体产生的感官信息做出适当的行为，因此每个强化学习智能体的状态空间随着在相同环境中操作的智能体的数量而指数增长。即使是简单的多智能体学习问题，通过单片的再聚焦学习方法也是计算困难的。我们将追捕问题的一个改进版本看作一个多智能体学习问题，并展示了模块化Q-学习猎物追捕智能体如何成功地合成捕获随机移动的猎物智能体所需的协调决策策略.多智能体学习通常是一个极其困难的问题，并且我们得到的结果强烈依赖于问题的特定属性.但结果是相当令人鼓舞的，表明我们的模块化强化学习方法在研究多个自治代理的自适应行为方面是有前途的。

In attempting to let artificial organisms or simple reactive robots synthesize some coordinated behavior, several researchers in the fields of artificial life and robotics have applied monolithic reinforcement-learning algorithms to multi-agent learning problems. In most of these applications, only a small number of learning agents are engaged in their joint tasks and accordingly the state space for each agent is relatively small. This is the reason why monolithic reinforcement-learning algorithms have been successfully applied to these multi-agent learning problems.However, these straightforward applications of reinforcement-learning algorithms do not successfully scale up to more complex multi-agent learning problems, where not a few learning agents are engaged in some coordinated tasks. In such a multi-agent problem domain, agents should appropriately behave according to not only sensory information produced by the physical environment itself but also that produced by other agents, and hence the state space for each reinforcement-learning agent grows exponentially in the number of agents operating in the same environment.Even simple multi-agent learning problems are computationally intractable by the monolithic reinfocrement-learning approaches. We consider a modified version of the Pursuit Problem as such a multi-agent learning problem, and show how successfully modular Q-learning prey-pursuing agents synthesize coordinated decision policies needed to capture a randomly-moving prey agent.Multi-agent learning is an extremely difficult problem in general, and the results we obtained strongly-rely on specific attributes of the problem. But the results are quite encouraging and suggest that our modular reinforcement-learning approach is promising in studying adaptive behavior of multiple autonomous agents.