Learning to manipulate cloth-like objects

由于这些物体的动力学复杂性，操纵诸如布（和绳子）之类的物体对机器人来说是具有挑战性的。例如，大多数这样的物体会表现出皱纹、折叠、皱缩或弯曲等现象——所有这些都很难精确建模，也很难实时预测。这也会导致感知问题，比如当布料的某些部分被自身遮挡时，就很难准确地观察和表示物体的当前状态。在这个领域工作，本研究的核心问题将包括以下问题：我们应该如何对这些对象的动态建模，以及哪些表示是易于学习和规划的？那么，为了在机器人系统上实际执行操作任务，这些表征应该如何与规划和控制相结合？第一个问题要求我们从定义适当的传感模式开始，以及对象表示——包括考虑基于分析或数值模拟的模型和基于数据驱动的机器学习的模型。在这项工作中，我们期望使用单目、立体和深度相机作为感知模式，以及各种结构化的基于神经网络的架构来解释这些数据。图形界已经广泛地研究了类布对象的模拟和可视化，尽管其中许多实现并不适合实时计算。此外，许多这样的模型更多地是由有效的可视化目标驱动的，而不是现实的物理行为，部分是作为应对计算限制的手段。我们将研究这些模型的变化，以及机器学习方法的使用——既可以有效地实时模拟物体动力学，也可以更好地校准模型到观测数据，以期随后的实时控制。本研究还将利用这些模型研究双手机器人的布料操作策略。这将包括基于模型的强化学习或模型预测控制体系结构的研究。这将包括对新的综合感知、规划和控制架构的研究，这些架构利用模型的结构特性来实现计算效率和行为的鲁棒性。本项目的实验将基于家庭和辅助用例。机器人行为的例子包括叠衣服、铺床、被窝和穿衣。在这个领域，挑战之一将是确保机器人能够安全地执行任务，同时补偿人类代理可能依次执行的动作。因此，评估整个系统的指标包括任务完成情况和人机交互效率。我们设想在Baxter机器人上进行初步实验，以及一个使用UR10手臂和多指手的定制双手机器人平台。这有利于多点抓取和动态操作，如投掷，拖拽等。

Manipulating objects such as cloth (and ropes) is challenging for robots due to the complexity of the dynamics of these objects. For instance, most such objects will exhibit phenomena including wrinkles, folds, and crumpling or buckling - all of which are hard to model exactly, and difficult to predict in real time. This also causes perceptual problems, such as when parts of the cloth are obscured by itself, making it difficult to accurately observe and represent the current state of the object.Working in this domain, the core concerns of this research will include the following questions: How should we model the dynamics of such objects and what representations are tractable for learning and planning? Then, how should these representations be integrated with planning and control, with a view to practical implementation of manipulation tasks on robotic systems?The first question requires us to start from defining appropriate sensing modalities, along with object representations - including consideration of analytical or numerical simulation-based models and data-driven machine learning based models. In this work, we expect to use monocular, stereo and depth cameras as the perceptual modalities, and various structured neural network-based architectures for the interpretation of this data.The graphics community have extensively studied simulation and visualisation of cloth-like objects, although many of these implementations are not suited for real-time computation. Furthermore, many such models are driven more by objectives of efficient visualisation rather than realistic physical behaviour, in part as a means of coping with computational limitations. We will investigate variations of such models, alongside the use of machine learning methods - both for efficient re-al-time emulation of the object dynamics and for better calibration of models to observational data, with a view to subsequent real-time control.This research will also investigate cloth manipulation strategies for bi-manual robots, using such models. This will include investigation of model-based reinforcement learning or model-predictive control architectures. This will include investigation into new integrated perception, planning and control architectures that leverage structural properties of the models to achieve computational efficiency and robustness of behaviour.Experiments in this project will be based on domestic and assistive use cases. Example robot behaviours would be folding clothes, making a bed, tucking into bed and dressing. In this domain, one of the challenges will be to ensure that the robot can perform its task safely whilst compensating for the actions that a human agent may perform in turn. So, the metrics on which the overall system could be evaluated include task achievement and efficiency of human-robot interaction. We envision initial experiments on the Baxter robot, and a custom bi-manual robot platform using UR10 arms and multi-fingered hands. This facilitates multi-point grasping and dynamic manipulations such as flinging, tugging, etc.