PHYDL: Physics-informed Differentiable Learning for Robotic Manipulation of Viscous and Granular Media

PHYDL：用于粘性和颗粒介质机器人操作的物理信息微分学习

• 批准号: EP/X018962/1
• 负责人: Ze Ji
• 金额: $ 25.61万
• 依托单位: CARDIFF UNIVERSITY
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FX018962%2F1
• 关键词: PHYDL; Physics; informed; Differentiable; Learning

Viscous and granular media are ubiquitous in our daily life, ranging from food like dough or beans to construction materials like concrete, soils, or sand. Humans can use their hands to transform sands to any shape with ease; cooks can manipulate dough while cooking with proper tools; construction workers can control various types of machinery to mine, transfer or pile rocks and soil. Apart from that, there exist many other activities that involve manipulating granular media, including disaster rescue, space exploration, underwater exploration, agriculture and so forth. As a result, improving techniques that can enable automatic manipulation of such substances in an applicable way is rewarding for many parts of our society, but, currently, it is a big challenge in robotic control.Conventional robot motion planning for manipulation focuses on safe and optimal trajectory generation with the assumption of rigid bodies of objects in the environment, that is, objects can only move or rotate but not deform. The intricate deformable geometric features and consequently the high unpredictability of material deformation due to the viscosity and granularity would prohibit direct applications of traditional robotic motion planning that is usually not scalable for such problems due to the requirement of explicitly designed models for rigid bodies. Techniques based on deep artificial neural networks and learning through intelligent agents interacting with the environment to achieve specific goals, known as Deep Reinforcement Learning (DRL), have become more popular for motion planning and decision making in complex environments without explicitly modelling the environment. DRL trains an agent or a robot by rewarding desired behaviours and/or punishing undesired ones, such that the DRL agent will learn to interpret its environment perception and take optimal actions through trial and error. Usually, the DRL agent is trained in a realistic simulation environment without deploying a real robot to interact with the real-world environment directly. However, most simulators only support rigid-body environments. On the other hand, numerical modelling for simulating such materials is usually computationally prohibitive and impractical for efficient DRL. Moreover, DRL requires a robot or agent to explore the environment with a large number of randomly selected actions in order to learn from getting rewards or penalties that are usually highly inefficient and unsafe.To address the above issues, this project will, for the first time, unlock a transformative robot learning framework by introducing a new technique, named differentiable physics, into the learning and control loop of the robot agent. This differentiable physics-based numerical simulation would greatly accelerate the simulation process, while on the other hand allow us to directly compute optimal physically-plausible actions without exploring all possible actions that are infinitely unbounded. In other words, we will leverage the differentiability nature for calculating physically-plausible bounded actions, which will reduce the amount of randomness for action exploration and hence allow a robot to learn more efficiently. This recent tendency has attracted increasing attention in different communities such as robot trajectory planning and differentiable physics. This project will unlock a new robot learning framework for highly efficient, physically-plausible, and safe deep reinforcement learning for autonomous robots to learn to manipulate viscous and granular materials.

粘性和颗粒状媒体在我们的日常生活中无处不在，从面团或豆类等食物到混凝土，土壤或沙子等建筑材料。人类可以轻松地用手将沙子转化为任何形状。厨师可以用适当的工具烹饪时操纵面团；建筑工人可以控制各种类型的机械，以挖掘，转移或桩岩和土壤。除此之外，还有许多其他活动涉及操纵颗粒状媒体，包括灾难救援，太空探索，水下探索，农业等。结果，改进能够以适用方式自动操纵这种物质的技术对我们社会的许多地方有所回报，但是目前，在机器人控制中，这是一个巨大的挑战。确定的机器人运动计划的操作集中在安全且最佳的轨迹生成上，而不是在环境中的固定物体的假设，而不是移动或旋转，而是旋转的。复杂的变形几何特征，因此由于粘度和粒度而导致的材料变形的高度不可预测性将禁止传统机器人运动计划的直接应用，由于需要明确设计的模型用于刚体，因此通常无法扩展此类问题。基于深层人工神经网络的技术和通过与环境相互作用以实现特定目标的智能代理（DRL）的技术，在复杂环境中的运动计划和决策方面已经变得更加流行，而无需显式建模环境。 DRL通过奖励所需的行为和/或惩罚不希望的行为来训练代理商或机器人，以便DRL代理人将学会解释其环境感知并通过反复试验和错误采取最佳行动。通常，DRL代理在现实的仿真环境中进行训练，而无需部署真正的机器人直接与现实世界环境进行交互。但是，大多数模拟器仅支持刚体的环境。另一方面，用于模拟此类材料的数值建模通常在计算上是过时的，并且对于有效的DRL来说是不切实际的。此外，DRL要求机器人或代理商以大量随机选择的操作来探索环境，以便从通常高效且不安全的奖励或罚款中学习，以解决上述问题，该项目将首次通过引入一个新型的物理学，介绍一个可识别的物理学，并将其命名为“学习和控制型”的机器人，这将首次释放一个变革性的机器人学习框架。这种基于物理学的数值模拟将大大加速模拟过程，而另一方面，我们可以直接计算最佳的物理上可行的动作，而无需探索所有无限无限的可能动作。换句话说，我们将利用与物理上可行的有限动作计算的可不同性质，这将减少动作探索的随机性，因此可以使机器人更有效地学习。这种最近的趋势吸引了越来越多的社区的关注，例如机器人轨迹计划和可区分的物理学。该项目将解锁一个新的机器人学习框架，以实现高效，物理上的和安全的深度加固学习，以便为自主机器人学习操纵粘性和颗粒状材料。

项目成果

Deep Reinforcement Learning With Explicit Context Representation

具有显式上下文表示的深度强化学习

• DOI: 10.1109/tnnls.2023.3325633
• 发表时间: 2023
• 期刊: IEEE Transactions on Neural Networks and Learning Systems
• 影响因子: 10.4
• 作者: Munguia-Galeano F

GAM: General affordance-based manipulation for contact-rich object disentangling tasks

GAM：针对接触丰富的对象解开任务的基于通用可供性的操作

• DOI: 10.1016/j.neucom.2024.127386
• 发表时间: 2024
• 期刊: Neurocomputing
• 影响因子: 6
• 作者: Yang X

Recent Advances of Deep Robotic Affordance Learning: A Reinforcement Learning Perspective

• DOI: 10.1109/tcds.2023.3277288
• 发表时间: 2023-03
• 期刊: IEEE Transactions on Cognitive and Developmental Systems
• 影响因子: 5
• 作者: Xintong Yang;Ze Ji;Jing Wu;Yunyu Lai