Robotic picking and packing with physical reasoning

具有物理推理能力的机器人拣选和包装

• 批准号: EP/V052659/1
• 负责人: Mehmet Dogar
• 金额: $ 152.5万
• 依托单位: University of Leeds
• 依托单位国家: 英国
• 项目类别: Fellowship
• 财政年份: 2021
• 资助国家: 英国
• 起止时间: 2021 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FV052659%2F1
• 关键词: Robotic; picking; packing; physical; reasoning

This Fellowship focuses on robotic object manipulation. Object manipulation refers to all the different ways robots can interact with objects in their environments. Consider the example of packing different items into a box in a warehouse to ship the box to a customer. To perform this, a robot would need to pick and insert the items into the box one by one, while nudging, pushing and squeezing objects, to achieve a tight packing. The robot would need to plan and control its actions, as well as use sensors to estimate the positions and deformations of the objects in the box.The dominant approach to object manipulation in the literature is geometry-based, where the world is represented using shapes and configurations only. While this simplifies planning and control, it also results in robots that are extremely limited in their skills. The central vision of this Fellowship is to go beyond that, by enabling robots to reason about, plan in, and control the full physics of the world. This has the potential to transform robots' object manipulation skills and our lives, because robots will be able to perform a much diverse variety of object manipulation skills applicable to manufacturing, assembly, and services.This Fellowship will create fundamental algorithms --- algorithms that can be applied to different object manipulation problems by other researchers and engineers. However, this Fellowship will also target a particular application area: picking and packing of objects for warehouse automation. With the rapid advance of e-commerce over the past decade, there is a pressing need to have efficient warehouse automation systems, in the UK and the world. However, existing robotic systems do not have physics-based reasoning, which limits their applications drastically. The physics-based picking and packing approach that I propose will enable robots to reach into cluttered bins/shelves/bags, pushing, nudging, and squeezing arbitrary objects to search and retrieve a particular object or to pack multiple objects tightly for shipping --- skills that do not exist in any current system. There are significant challenges to using physics-based models during robotic manipulation. An important one is computational expense. We have low-level physics models and physics engines, similar to the ones used in computer games, which can be used by robots. However, computing such models are expensive (i.e. takes too much computer time) and robotic algorithms need to query such models thousands, and sometimes millions, of times before choosing an action, making it infeasible to use such a straightforward approach. Instead, I propose to develop and use hierarchical models of physics. At higher levels in this hierarchy are coarse, approximate physics models, i.e. models that are computationally cheap (i.e. fast to compute) but may be inaccurate. At lower levels in this hierarchy are fine models, i.e. models that are computationally expensive (i.e. slow to compute) but are accurate. I will investigate a variety of methods (including data-driven methods as well as parallel computing methods) to learn and compute such a hierarchy of physics models. I will also develop new planning, control, and state estimation algorithms that can use these new hierarchical physics models. I will also use these new algorithms and systems to accelerate the adoption of this technology in the UK. I will work with the EPSRC UK Robotics & Autonomous Systems Network and industrial stakeholders to develop a roadmap for the integration of autonomous picking and packing robots into the existing industrial workflows. I will also aim to create a new national organisation to focus on this important technology.To achieve these aims, I will work with many academic and industrial partners, including the Advanced Supply Chain Group, a leading UK-based supply chain and warehouse management company, as well as a key international company in this area, Amazon.

该奖学金专注于机器人对象操作。对象操纵指的是机器人与其环境中的对象交互的所有不同方式。考虑一下这样一个示例：将不同的物品打包到仓库中的一个盒子中，然后将该盒子运送给客户。要做到这一点，机器人需要一个接一个地挑选物品并将其插入盒子，同时轻推和挤压物品，以实现紧密包装。机器人需要计划和控制它的动作，并使用传感器来估计盒子中物体的位置和变形。文献中操纵物体的主要方法是基于几何的，其中世界只使用形状和配置来表示。虽然这简化了计划和控制，但也导致机器人的技能极其有限。这个联谊会的中心愿景是超越这一点，使机器人能够推理、规划和控制世界的全部物理。这有可能改变机器人的对象操作技能和我们的生活，因为机器人将能够执行适用于制造、组装和服务的各种对象操作技能。该奖学金将创建基本算法-可由其他研究人员和工程师应用于不同对象操作问题的算法。然而，该奖学金还将针对一个特定的应用领域：仓库自动化物品的挑选和包装。随着电子商务在过去十年中的快速发展，在英国和世界上，迫切需要高效的仓库自动化系统。然而，现有的机器人系统没有基于物理的推理，这极大地限制了它们的应用。我提出的基于物理的挑选和打包方法将使机器人能够伸手进入杂乱无章的垃圾箱/货架/袋子，推动、轻推和挤压任意物体，以搜索和取回特定物体或将多个物体紧紧地打包以供运输-这是任何当前系统中都不存在的技能。在机器人操作过程中使用基于物理的模型存在着巨大的挑战。一个重要的问题是计算费用。我们有低级的物理模型和物理引擎，类似于计算机游戏中使用的，可以被机器人使用。然而，计算这样的模型是昂贵的(即花费太多的计算机时间)，并且机器人算法在选择动作之前需要查询这样的模型数千次，有时甚至数百万次，这使得使用这种直接的方法是不可行的。相反，我建议开发和使用物理的分层模型。在此层次结构中的较高级别是粗略、近似的物理模型，即计算成本较低(即计算速度较快)但可能不准确的模型。在此层次结构的较低级别是精细模型，即计算成本较高(即计算速度较慢)但准确的模型。我将研究学习和计算这种物理模型层次结构的各种方法(包括数据驱动方法和并行计算方法)。我还将开发新的规划、控制和状态估计算法，这些算法可以使用这些新的分层物理模型。我还将使用这些新的算法和系统来加快这项技术在英国的采用。我将与EPSRC UK Robotics&Automatic Systems Network和行业利益相关者合作，制定将自动挑选和包装机器人整合到现有工业工作流程中的路线图。我还将致力于创建一个新的全国性组织，专注于这项重要的技术。为了实现这些目标，我将与许多学术和工业合作伙伴合作，包括总部位于英国的领先供应链和仓库管理公司Advanced Supply Chain Group，以及这一领域的关键国际公司亚马逊。

项目成果

To ask for help or not to ask: A predictive approach to human-in-the-loop motion planning for robot manipulation tasks

寻求帮助还是不寻求帮助：用于机器人操作任务的人在环运动规划的预测方法

• DOI: 10.1109/iros47612.2022.9981679
• 发表时间: 2022
• 作者: Papallas R

Learning to Efficiently Plan Robust Frictional Multi-Object Grasps

学习有效地规划稳健的摩擦多物体抓取

• DOI: 10.1109/iros55552.2023.10341895
• 发表时间: 2023
• 作者: Agboh W

The Busboy Problem: Efficient Tableware Decluttering Using Consolidation and Multi-Object Grasps

勤杂工问题：使用整合和多对象抓取来高效整理餐具

• DOI: 10.1109/case56687.2023.10260474
• 发表时间: 2023
• 作者: Srinivas K

Goal-Conditioned Action Space Reduction for Deformable Object Manipulation

可变形物体操纵的目标条件动作空间缩减

• DOI: 10.1109/icra48891.2023.10161541
• 发表时间: 2023
• 作者: Wang S

Robotics Research

• DOI: 10.1007/978-3-031-25555-7
• 发表时间: 2023
• 期刊: Robotics Research