Deep Reinforcement Learning for Dynamic Locomotion of Humanoid Robots

该研究项目的重点是使用深度强化学习来解决人形机器人的动态运动问题。在过去，运动主要是通过传统的分析方法来完成的，例如模型预测控制，因为它们需要人力和知识，并且需要高计算能力才能在线运行，因此受到限制。与在线计算方法相比，机器学习方法的计算可以离线外包。通过这样做，可以实现高维控制系统（如类人系统）更快的在线性能。与分析方法相比，深度强化学习等机器学习方法也需要更少的人力来设计。随着深度强化学习算法的日益强大，越来越多的研究工作使用深度强化学习来解决控制任务，因为针对连续动作域设计的深度强化学习算法的最新进展为应用涉及复杂动力学的强化学习连续控制任务提供了可能。该项目计划探索使用深度强化学习获得与分析方法相当或更好的双足控制策略的可行性，同时使用更少的人力。并最终设计一个基于深度强化学习的控制框架，该框架能够在最小的人为干预下学习广泛的平衡和行走策略，并将设计的控制框架的性能与当前人类劳动密集型分析工程方法进行比较。该项目还将探索与强化学习相关的其他方法，如模仿学习和迁移学习，并将其实施到拟议的控制框架中。

The research project is focused on using deep reinforcement learning tosolve dynamic locomotion of humanoid robots. In the past, locomotion ismainly done using conventional analytical approaches, e.g.Model-Predictive Control, which are limited because they require humaneffort and knowledge, and demand high computing power to run online. Incontrast to on-line computation methods, the computation for machinelearning approaches can be outsourced offline. By doing so, fasteronline performance for high dimensional control systems, such ashumanoids, can be achieved. Machine learning approaches such as deepreinforcement learning also requires less human effort to designcompared to analytical approaches.Given the increasingly more powerful deep RL algorithms, an increasingnumber of research works have used deep RL to solve control tasks, asthe recent progress in deep RL algorithms designed for continuous actiondomain has brought forward the possibility to apply reinforcementlearning continuous control tasks that involve complicated dynamics.The project plans to explore the feasibilities of using deepreinforcement learning to acquire bipedal control policies comparable orbetter than analytical approaches while using less human effort. Andeventually design a control framework based on deep RL that is capableof learning a wide range of balancing and walking strategies withminimum human intervention, and compare the performance of the designedcontrol framework with current human labour intensive analyticengineering approach. The project will also explore other methodologiesrelated to reinforcement learning, such as imitation learning andtransfer learning, and implement them into the proposed controlframework.