PHYDL: Physics-informed Differentiable Learning for Robotic Manipulation of Viscous and Granular Media

PHYDL：用于粘性和颗粒介质机器人操作的物理信息微分学习

• 批准号: EP/X018962/1
• 负责人: Ze Ji
• 金额: $ 25.61万
• 依托单位: CARDIFF UNIVERSITY
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FX018962%2F1
• 关键词: PHYDL; Physics; informed; Differentiable; Learning

Viscous and granular media are ubiquitous in our daily life, ranging from food like dough or beans to construction materials like concrete, soils, or sand. Humans can use their hands to transform sands to any shape with ease; cooks can manipulate dough while cooking with proper tools; construction workers can control various types of machinery to mine, transfer or pile rocks and soil. Apart from that, there exist many other activities that involve manipulating granular media, including disaster rescue, space exploration, underwater exploration, agriculture and so forth. As a result, improving techniques that can enable automatic manipulation of such substances in an applicable way is rewarding for many parts of our society, but, currently, it is a big challenge in robotic control.Conventional robot motion planning for manipulation focuses on safe and optimal trajectory generation with the assumption of rigid bodies of objects in the environment, that is, objects can only move or rotate but not deform. The intricate deformable geometric features and consequently the high unpredictability of material deformation due to the viscosity and granularity would prohibit direct applications of traditional robotic motion planning that is usually not scalable for such problems due to the requirement of explicitly designed models for rigid bodies. Techniques based on deep artificial neural networks and learning through intelligent agents interacting with the environment to achieve specific goals, known as Deep Reinforcement Learning (DRL), have become more popular for motion planning and decision making in complex environments without explicitly modelling the environment. DRL trains an agent or a robot by rewarding desired behaviours and/or punishing undesired ones, such that the DRL agent will learn to interpret its environment perception and take optimal actions through trial and error. Usually, the DRL agent is trained in a realistic simulation environment without deploying a real robot to interact with the real-world environment directly. However, most simulators only support rigid-body environments. On the other hand, numerical modelling for simulating such materials is usually computationally prohibitive and impractical for efficient DRL. Moreover, DRL requires a robot or agent to explore the environment with a large number of randomly selected actions in order to learn from getting rewards or penalties that are usually highly inefficient and unsafe.To address the above issues, this project will, for the first time, unlock a transformative robot learning framework by introducing a new technique, named differentiable physics, into the learning and control loop of the robot agent. This differentiable physics-based numerical simulation would greatly accelerate the simulation process, while on the other hand allow us to directly compute optimal physically-plausible actions without exploring all possible actions that are infinitely unbounded. In other words, we will leverage the differentiability nature for calculating physically-plausible bounded actions, which will reduce the amount of randomness for action exploration and hence allow a robot to learn more efficiently. This recent tendency has attracted increasing attention in different communities such as robot trajectory planning and differentiable physics. This project will unlock a new robot learning framework for highly efficient, physically-plausible, and safe deep reinforcement learning for autonomous robots to learn to manipulate viscous and granular materials.

粘性和颗粒状介质在我们的日常生活中无处不在，从面团或豆类等食物到混凝土，土壤或沙子等建筑材料。人类可以用双手轻松地将沙子变成任何形状;厨师可以在烹饪时使用适当的工具操纵面团;建筑工人可以控制各种类型的机械来开采，运输或堆放岩石和土壤。除此之外，还有许多其他活动涉及操纵颗粒介质，包括灾难救援，太空探索，水下探索，农业等。因此，改进技术，使这些物质的自动操作，以适用的方式是我们的社会的许多部分的回报，但目前，这是一个很大的挑战，在机器人控制。传统的机器人运动规划的操作集中在安全和最佳的轨迹生成与假设的刚体的对象在环境中，即，对象只能移动或旋转，但不能变形。复杂的可变形的几何特征，因此，由于粘性和粒度的材料变形的高度不可预测性将禁止直接应用传统的机器人运动规划，这通常是不可扩展的，由于明确设计的模型的刚性体的要求，这样的问题。基于深度人工神经网络和通过智能代理与环境交互以实现特定目标的学习技术（称为深度强化学习（DRL））在复杂环境中的运动规划和决策制定中变得更加流行，而无需显式地对环境建模。DRL通过奖励期望的行为和/或惩罚不期望的行为来训练代理或机器人，使得DRL代理将学习解释其环境感知并通过试错来采取最佳行动。通常，DRL智能体是在真实的仿真环境中训练的，而不需要部署真实的机器人与真实世界环境直接交互。然而，大多数模拟器只支持刚体环境。另一方面，用于模拟这种材料的数值建模通常在计算上是禁止的，并且对于有效的DRL是不切实际的。此外，DRL要求机器人或智能体通过大量随机选择的动作来探索环境，以便从通常非常低效和不安全的奖励或惩罚中学习。为了解决上述问题，该项目将首次通过引入一种名为可微物理的新技术来解锁一个变革性的机器人学习框架，进入机器人智能体的学习和控制回路。这种基于可微物理的数值模拟将大大加快模拟过程，而另一方面，使我们能够直接计算最佳的物理合理的行动，而无需探索所有可能的行动是无限无界的。换句话说，我们将利用可微性来计算物理上合理的有界动作，这将减少动作探索的随机性，从而使机器人能够更有效地学习。这一趋势在机器人轨迹规划和可微物理等领域引起了越来越多的关注。该项目将解锁一个新的机器人学习框架，用于高效、物理上合理和安全的深度强化学习，使自主机器人学习操纵粘性和颗粒状材料。

项目成果

Deep Reinforcement Learning With Explicit Context Representation

具有显式上下文表示的深度强化学习

• DOI: 10.1109/tnnls.2023.3325633
• 发表时间: 2023
• 期刊: IEEE Transactions on Neural Networks and Learning Systems
• 影响因子: 10.4
• 作者: Munguia-Galeano F

GAM: General affordance-based manipulation for contact-rich object disentangling tasks

GAM：针对接触丰富的对象解开任务的基于通用可供性的操作

• DOI: 10.1016/j.neucom.2024.127386
• 发表时间: 2024
• 期刊: Neurocomputing
• 影响因子: 6
• 作者: Yang X

Recent Advances of Deep Robotic Affordance Learning: A Reinforcement Learning Perspective

• DOI: 10.1109/tcds.2023.3277288
• 发表时间: 2023-03
• 期刊: IEEE Transactions on Cognitive and Developmental Systems
• 影响因子: 5
• 作者: Xintong Yang;Ze Ji;Jing Wu;Yunyu Lai