Learning Loco-Manipulation for Articulated Robots

腿机器人技术的进步将使广泛的应用成为可能，从行星和洞穴探索，到穿越危险环境进行搜索和救援行动，并通过自主环境监测受益于保护工作。在英国，一直有强烈的动机部署腿式机器人来检查核电站和海上发电厂，目的是将人类工人在危险和偏远环境中的风险降至最低。在家庭环境中，这类机器人具有提高安全性、便利性和生活质量的潜力，帮助完成清洁和安保等任务，并为老年人或行动不便的人提供帮助。腿式机器人最大的挑战之一在于使它们能够穿越我们周围复杂和快速变化的环境。在这个研究项目中，我们将通过使用机器学习，特别是强化学习(RL)来解决这个问题。通过允许机器人学习它们的经验，我们可以极大地提高它们的性能和对周围日常世界的动态性质的鲁棒性。本项目的主要目标是开发能够极大地增强腿部机器人的运动和穿越能力的控制方法。为此，我们将研究腿部机器人如何学习接受周围环境的信息，以适应周围发生的不确定性和变化，即所谓的短期运动规划。这可以提高它们的敏捷性和平衡性，使它们能够在更广泛的恶劣室外和室内环境中安全行走。在自然界中，动物不仅对突然的干扰做出反应，而且可以主动预测干扰，并先发制人地修改它们的计划和调整它们的运动方式。例如，通过感官和过往知识的结合，人类可以预测到光滑的表面，并在滑动发生之前调整他们的步子以避免打滑。此外，当穿越高度不规律的环境时，反应行为是不够的，这需要更高水平的远见和长期规划，提前几秒钟甚至几分钟。要执行高度灵活的运动，不仅需要纯粹的反射行为，还需要对机器人周围世界的属性进行更多的推理，以及形成复杂运动计划的能力。为了应对这一挑战，我们将采用一种名为深度强化学习(DRL)的新方法。DRLearning通过基于模拟的神经网络训练来利用大数据来设计更灵活、更健壮的运动控制器。它允许这样的管制员从他们的错误和经验中吸取教训，并不断改进。最近的研究表明，与传统方法相比，通过RL腿部机器人可以被训练出穿越更加多样化的户外地形。在这个项目中，我们将RL应用到更复杂的环境中，遍历这些环境需要高级决策、远见和对世界的感知相结合。整个项目的重点将是在真实机器人上部署和测试这些运动控制器。这样，我们可以确保我们的控制器足够强大和灵活，可以离开实验室并在现实世界中使用。确保这些系统的安全，特别是在人类在场的情况下运行，这是至关重要的，在现场仍然是一个悬而未决的问题。因此，作为我们研究问题的一部分，我们将进一步研究基于学习的控制器如何纳入可解释性和安全性考虑因素。本项目属于EPSRC人工智能和机器人研究领域。作为一个工业案例项目的一部分，我们将与工业合作伙伴戴森密切合作。这可以帮助弥合学术研究和行业之间的差距，并促进可以稳健和安全地部署的系统的开发。

Advancements in legged robot technology will enable a wide range of applications, from planetary and cave exploration, to traversing dangerous environments for search and rescue operations and benefiting conservation efforts through autonomous environmental monitoring. In the UK there has been a strong incentive towards deploying legged robots for inspection of nuclear and offshore power plants, with the goal of minimising the exposure of human workers to risks in hazardous and remote settings. In a domestic setting such robots have the potential the improve safety, convenience and quality of life, helping with tasks tasks like cleaning and security, and providing assistance to elderly or mobility-impaired people.One of the greatest challenges with legged robots lies in enabling them to traverse the complex and rapidly changing environments around us. In this research project we will tackle this problem through the use of machine learning, and in particular Reinforcement Learning (RL). By allowing robots to learn from their experience we can greatly improve their performance and robustness to the dynamic nature of the everyday world around them.The main goal of this project is to develop methods of control that can greatly enhance the locomotion and traversal capabilities of legged robots. To this end, we would investigate how a legged robot can learn to take in information from its surroundings to adapt to the uncertainty and changes that occur around it, known as short-horizon motion planning. This can boost their agility and balance, and allow them to safely walk through a wider range of harsh outdoor and indoor environments alike.In nature, animals not only react to sudden disruptions but can actively anticipate disturbances and preemptively modify their plans and adapt their locomotion. As an example, through a combination of their senses and past knowledge, humans can anticipate a slippery surface and adjust their step to avoid slipping before it happens. Moreover, reactive behaviour is not sufficient when traversing highly irregular environments, which necessitate a greater level of foresight and planning in the long-term, several seconds or even minutes ahead. To perform highly agile motions requires not only purely reflexive behaviour, but a greater deal of reasoning about the properties of the world around the robot, and the ability to form complex motion plans.To tackle this challenge we will employ a novel approach known as Deep Reinforcement Learning (DRL). DRLearning leverages big data through simulation-based training of neural networks to design much more agile and robust motion controllers. It allows such controllers to learn from their mistakes and their experience and continuously improve. Recent works have shown that through RL legged robots can be taught to traverse a much more diverse set of outdoor terrains than traditional methods. In this project we would apply RL to much more complex environments, the traversal of which requires a combination of advanced decision-making, foresight and perception of the world. A strong emphasis throughout the project will be the deployment and testing of these motion controllers on real robots. This way we can ensure that our controllers are robust and flexible enough to leave the lab and be used in the real world. Ensuring the safety of these systems, especially when operating in the presence of humans, is crucial and remains an open problem in the field. Therefore, as part of our research questions we will further investigate how learning-based controllers can incorporate interpretability and safety considerations.This project falls within the EPSRC Artificial intelligence and Robotics research area. As part of an industrial CASE project, we will work closely with Dyson, the industrial partner. This can help bridge the gap between academic research and industry, and promote the development of systems that can be robustly and safely deployed.