Robotic picking and packing with physical reasoning

本奖学金的重点是机器人对象操作。对象操作是指机器人与环境中的对象交互的所有不同方式。考虑这样一个例子：将不同的物品打包到仓库中的一个盒子中，然后将盒子运送给客户。要做到这一点，机器人需要将物品一件一件地取出并插入盒子，同时轻推、推和挤压物体，以实现紧密包装。机器人需要计划和控制自己的行动，并使用传感器来估计盒子中物体的位置和变形。在文献中，对象操作的主要方法是基于几何的，其中世界仅使用形状和配置来表示。虽然这简化了计划和控制，但也导致机器人的技能极其有限。这个项目的核心愿景是超越这一点，让机器人能够推理、计划和控制世界的全部物理。这有可能改变机器人的物体操作技能和我们的生活，因为机器人将能够执行适用于制造、装配和服务的各种各样的物体操作技能。该奖学金将创建基本算法——可以被其他研究人员和工程师应用于不同对象操作问题的算法。然而，这个奖学金也将针对一个特定的应用领域：仓库自动化的拣选和包装对象。随着电子商务在过去十年的快速发展，在英国和世界各地都迫切需要高效的仓库自动化系统。然而，现有的机器人系统没有基于物理的推理，这极大地限制了它们的应用。我提出的基于物理的拣选和包装方法将使机器人能够进入杂乱的垃圾箱/货架/袋子，推动，轻推和挤压任意物体以搜索和检索特定物体或将多个物体紧密打包以进行运输-这些技能在任何当前系统中都不存在。在机器人操作过程中使用基于物理的模型存在重大挑战。一个重要的问题是计算费用。我们有低级的物理模型和物理引擎，类似于电脑游戏中使用的，可以被机器人使用。然而，计算这样的模型是昂贵的（即需要太多的计算机时间），机器人算法需要在选择一个动作之前查询这样的模型数千次，有时甚至数百万次，使得使用这种直接的方法是不可行的。相反，我建议发展和使用物理的层次模型。在这个层次结构的更高层次是粗糙的、近似的物理模型，即计算成本低（即计算速度快）但可能不准确的模型。在这个层次结构中较低的层次是精细模型，即计算成本高（即计算速度慢）但精度高的模型。我将研究各种方法（包括数据驱动的方法以及并行计算方法）来学习和计算这种物理模型的层次结构。我还将开发新的计划、控制和状态估计算法，这些算法可以使用这些新的分层物理模型。我还将使用这些新的算法和系统来加速这项技术在英国的采用。我将与EPSRC英国机器人与自主系统网络以及工业利益相关者合作，制定将自主采摘和包装机器人集成到现有工业工作流程中的路线图。我还将致力于创建一个新的国家组织，专注于这项重要的技术。为了实现这些目标，我将与许多学术和工业合作伙伴合作，包括英国领先的供应链和仓库管理公司Advanced Supply Chain Group，以及该领域的主要国际公司亚马逊。

This Fellowship focuses on robotic object manipulation. Object manipulation refers to all the different ways robots can interact with objects in their environments. Consider the example of packing different items into a box in a warehouse to ship the box to a customer. To perform this, a robot would need to pick and insert the items into the box one by one, while nudging, pushing and squeezing objects, to achieve a tight packing. The robot would need to plan and control its actions, as well as use sensors to estimate the positions and deformations of the objects in the box.The dominant approach to object manipulation in the literature is geometry-based, where the world is represented using shapes and configurations only. While this simplifies planning and control, it also results in robots that are extremely limited in their skills. The central vision of this Fellowship is to go beyond that, by enabling robots to reason about, plan in, and control the full physics of the world. This has the potential to transform robots' object manipulation skills and our lives, because robots will be able to perform a much diverse variety of object manipulation skills applicable to manufacturing, assembly, and services.This Fellowship will create fundamental algorithms --- algorithms that can be applied to different object manipulation problems by other researchers and engineers. However, this Fellowship will also target a particular application area: picking and packing of objects for warehouse automation. With the rapid advance of e-commerce over the past decade, there is a pressing need to have efficient warehouse automation systems, in the UK and the world. However, existing robotic systems do not have physics-based reasoning, which limits their applications drastically. The physics-based picking and packing approach that I propose will enable robots to reach into cluttered bins/shelves/bags, pushing, nudging, and squeezing arbitrary objects to search and retrieve a particular object or to pack multiple objects tightly for shipping --- skills that do not exist in any current system. There are significant challenges to using physics-based models during robotic manipulation. An important one is computational expense. We have low-level physics models and physics engines, similar to the ones used in computer games, which can be used by robots. However, computing such models are expensive (i.e. takes too much computer time) and robotic algorithms need to query such models thousands, and sometimes millions, of times before choosing an action, making it infeasible to use such a straightforward approach. Instead, I propose to develop and use hierarchical models of physics. At higher levels in this hierarchy are coarse, approximate physics models, i.e. models that are computationally cheap (i.e. fast to compute) but may be inaccurate. At lower levels in this hierarchy are fine models, i.e. models that are computationally expensive (i.e. slow to compute) but are accurate. I will investigate a variety of methods (including data-driven methods as well as parallel computing methods) to learn and compute such a hierarchy of physics models. I will also develop new planning, control, and state estimation algorithms that can use these new hierarchical physics models. I will also use these new algorithms and systems to accelerate the adoption of this technology in the UK. I will work with the EPSRC UK Robotics & Autonomous Systems Network and industrial stakeholders to develop a roadmap for the integration of autonomous picking and packing robots into the existing industrial workflows. I will also aim to create a new national organisation to focus on this important technology.To achieve these aims, I will work with many academic and industrial partners, including the Advanced Supply Chain Group, a leading UK-based supply chain and warehouse management company, as well as a key international company in this area, Amazon.