Study on Movement-Pattern Acquisition of Biological Robot using Reinforcement Learning from Evolutional viewpoint

本研究从进化的角度出发，利用强化学习方法对生物机器人的运动模式获取进行了阐述。通过昆虫和爬行动物的比较，脊椎是实现小能量运动的关键因素。对于哺乳动物而言，其运动模式的对称条件减少了能量消耗较小的向前运动模式的数量。其次，基于概率的运动模式可视化方法，明确了无马尔可夫性质的巨摆机器人的各种奖励与巨摆成功的关系。最后，主观奖励比客观奖励有更好的表现，因为尽管客观奖励是从当前条件获得的，但人类可以估计从过去到未来的一系列运动模式。

This study clarifies movement-pattern acquisitions of biological robots using reinforcement learning method from evolutionary view point. By comparison between an insect and a reptile, a vertebral column gives essential factor to realize forward movements with small energy consumption. As for a mammal, symmetric condition of its movement-pattern decrease number of forward movement-patterns with small energy consumption. Next, visualization method of motion-pattern with probability clarifies relation between various rewards and success of giant swing for giant swing robot without Markov property. Finally, subjective rewards gives better performance than that of objective rewards because human being can estimate a sequence of movement-pattern form past to future although objective reward is obtained from a present condition.