Study on Decentralized Learning Algorithms in Markovian Environments

本文的主要研究成果如下：（1）针对动态未知的马尔可夫链，提出了一种新的分散学习算法。详细的仿真研究揭示了我们算法的可行性及其相对于Q学习方案的优越性。(2)我们提出了一个面向对象的设计支持系统开发自主移动的机器人。我们的支持系统的有用性进行了检查，通过一些模拟和真实的机器人的实现。通过使用这个支持系统，我们还开发了一个机器人，它可以获得一个适当的设置中的避障算法，称为VFH的增益因子，通过学习。(3)将分散学习算法应用于智能移动的机器人的自适应动作选择问题。我们使用了一个机器人，它有两个光电传感器来测量左右方向的光强。机器人的任务是学习一个动作选择策略，用于移动到放置在房间任何位置的灯。分散式学习方法通过运行模拟机器人成功地进行了测试，这些机器人是由面向对象的设计支持系统实现的。(4)提出了一类称为霍隆网络的递阶系统作为非线性动力系统辨识的一般模型。霍隆网络能够通过自组织其结构来进化，并且能够在不假设对它们有太多知识的情况下学习非线性系统。

The results of this study are summarized as follows :(1) We proposed a new decentralized learning algorithm for Markov chains with unknown dynamics. A detailed simulation study revealed the feasibility of our algorithm and its superiority to the Q-learning scheme.(2) We proposed an object-oriented design support system for developing autonomous mobile robots. The usefulness of our support system was examined through some implementations of simulated and real robots. By using this support system, we also developed a robot which can acquire a proper setting of gain factors in an obstacle avoidance algorithm, called the VFH,by learning.(3) We applied the decentralized learning algorithm to the problem of adaptive action selection in an intelligent mobile robot. We employed a robot which has two photosensors to measure the light intensity in right or left direction. The robot's task is to learn an action selection policy for moving toward and getting to a light placed in any location of a room. The decentralized learning approach was successfully tested by running simulated robots, which were implemented by the object-oriented design support system.(4) We proposed a class of hierarchical systems called holon networks as general models for identification of nonlinear dynamical systems. Holon networks are able to evolve by self-organizing their structure and learn nonlinear systems without assuming much knowledge of them.