Multiple-Strategy Based Multi-Agent Systems and Its Learning Method

研究的目的是提出一种反应性规划剂，该规划剂可以适应通常由隐藏的多个动态因素组合而产生的复杂环境。为此，我们开发了多策略方法，通过独立学习代理获得互补的反应策略，然后相互合作以实现全局目标。项目期间取得的成果总结如下：1。我们考虑了一个由单个代理包含多个子代理组成的体系结构，这些子代理具有部分输入状态向量，并且具有独立的性能标准和学习机制。对于子代理的自动任务分解和仲裁，我们为每个子代理引入了一个使用函数，该函数由成功率和最近的观察结果指定。结果表明，使用函数可以产生判别压力。然后，将使用函数的输出作为教学信号，将子代理最近的分配历史作为状态，引入学习仲裁模块。我们通过聚合问题解决的概念展示了另一种方法，它涉及到对代理的自动任务分解。为了给知识的分解提供压力，我们通过支付函数的相互交换引入了agent之间的交互，因为这是保证在任何强化学习算法中使用的唯一信息。为此，提出了一种收益过滤函数。结果表明，过滤函数的函数形式应根据整个智能体的目标函数设置为竞争或合作。本报告展示了该方法对移动机器人死锁避免问题的实际实现。结果表明了该算法的有效性和良好的性能。

The objective of the research is to propose a reactive planning agent that can adopt to the complex environment which usually is generated by the combination of hidden multiple dynamical factors. To this end, we have developed multi-strategic approach by independent learning agents that acquire complemental reactive strategies and then co-operate with each other to achieve global objective. The results obtained from the term of the project is summarised as follows :1. We have considered an architecture that consists of single agent containing multiple sub-agents, which have portion of input state vector and have independent performance criteria and learning mechanism. For the automatic task decomposition and arbitration of sub-agents, we have introduced a usage function for each sub-agent, which is specified by the success rate and the results of recent observations. It is shown that the usage function can be used to generate the discrimination pressure. Then by using the output of usage function as a teaching signal and the history of recent sub-agent allocation as a state, a learning arbitration module can be introduced.2. We have shown another approach by the notion of aggregate problem solving, which involves the automatic task decomposition to agents. Providing the pressure for the decomposition of knowledge, we have introduced interaction between agents through the mutual exchange of payoff function, because it is an only information that guarantees to be used in any reinforcement learning algorithm. A payoff filtering function is proposed to this end. It is shown that the function form of the filtering function should be set either competitive or co-operative according to the objective function of the whole agents. This report shows practical implementation of the method to a deadlock avoidance problem of mobile robots. The results show the effect and the promising property of the algorithm.