SELF-ORGANIZING MULTI-AGENT SYSTEMS : ARTIFICIAL LIFE APPROACHES

为了让人工生物或简单的反应机器人合成一些协调的行为，人工生命和机器人领域的一些研究人员已经将单片并行学习算法应用于多智能体学习问题。在大多数这些应用中，只有少量的学习代理参与他们的联合任务，因此每个代理的状态空间相对较小。这就是为什么单片并行学习算法已经成功地应用于这些多智能体学习问题的原因。然而，并行学习算法的这些直接应用不能成功地扩展到更复杂的多智能体学习问题，其中不是少数学习智能体参与一些协调的任务。在这样一个多智能体问题域中，智能体不仅要根据物理环境本身产生的感知信息，还要根据其他智能体产生的感知信息来适当地行为，因此每个自主学习智能体的状态空间随着在同一环境中运行的智能体数量的增加而呈指数增长。我们考虑一个修改版本的追求问题，这样一个多智能体学习问题，并成功地展示了模块化Q学习的捕食追求代理合成协调的决策政策需要捕捉一个随机移动的猎物agent.Multi-agent学习是一个非常困难的问题，我们得到的结果强烈依赖于特定的属性的问题。但结果是相当令人鼓舞的，并表明我们的模块化的重复学习方法是有前途的研究多个自治代理的自适应行为。

In attempting to let artificial organisms or simple reactive robots synthesize some coordinated behavior, several researchers in the fields of artificial life and robotics have applied monolithic reinforcement-learning algorithms to multi-agent learning problems. In most of these applications, only a small number of learning agents are engaged in their joint tasks and accordingly the state space for each agent is relatively small. This is the reason why monolithic reinforcement-learning algorithms have been successfully applied to these multi-agent learning problems.However, these straightforward applications of reinforcement-learning algorithms do not successfully scale up to more complex multi-agent learning problems, where not a few learning agents are engaged in some coordinated tasks. In such a multi-agent problem domain, agents should appropriately behave according to not only sensory information produced by the physical environment itself but also that produced by other agents, and hence the state space for each reinforcement-learning agent grows exponentially in the number of agents operating in the same environment.Even simple multi-agent learning problems are computationally intractable by the monolithic reinfocrement-learning approaches. We consider a modified version of the Pursuit Problem as such a multi-agent learning problem, and show how successfully modular Q-learning prey-pursuing agents synthesize coordinated decision policies needed to capture a randomly-moving prey agent.Multi-agent learning is an extremely difficult problem in general, and the results we obtained strongly-rely on specific attributes of the problem. But the results are quite encouraging and suggest that our modular reinforcement-learning approach is promising in studying adaptive behavior of multiple autonomous agents.