Sim-to-Real Deep Reinforcement Learning for legged robot locomotion with vision-based high dimensional data

点击翻译按钮获取中文摘要

This PhD will explore methods that allow legged robots to improve and adapt its gaits to various terrains. For the MSc the physics simulator was pre-programmed with the environment terrain and robot dimensions. Parameters such as friction coefficients and weight distributions were roughly estimated. However, for the PhD the robot will build a model of itself in a 3D environment using a combination of vision and depth sensing combined with orientation sensing and robot babbling. This will allow the robot to adjust the parameters of the simulated environment which may increase adaptability and reduce the reality gap.The aim is to contribute a novel method that allows any type of legged robot to manoeuvre from high dimensional input data. This method aims to address the adaptability problems with explicitly programmed algorithms whist also addressing the reality gap issues with PPO reinforcement learning. The input data will be from RGBD, joint parameters, orientation and tactile sensing. Note PPO performs exceptionally well with high dimension inputs and these will be required in order to identify the complexities of the real world.Aims and objectives:1. Build a legged robot capable of sensing its environment (i.e. RGBD, orientation and tactile sensors)2. Self-model Agent - Allow the robot to perform robot babbling and use the orientation, tactile and joint position sensors to self-model the agent3. Model Environment - Allow the robot to scan the room with RGBD sensors to model its terrain and identify its target location (i.e. a ball in a room)4. Train the simulated robot with reinforcement learning in the 3D world to determine a policy to reach the target location5. Deploy the trained policy on the physical robot 6. Measure the 'reality gap' between the robot's performance in simulation and the physical world.7. Adapt robot babbling and environment modelling accordingly.8. TRL 4 quadruped robot that can traverse previously unseen terrains/scenarios toward a goal