Learning to manipulate cloth-like objects

由于布料(和绳索)等物体动力学的复杂性，对机器人来说是具有挑战性的。例如，大多数这样的物体会表现出褶皱、折叠和褶皱或弯曲等现象--所有这些都很难准确建模，也很难实时预测。这也会导致感知问题，例如当布料的某些部分被自身遮挡时，使得很难准确地观察和表示对象的当前状态。在这一领域，本研究的核心关注将包括以下问题：我们应该如何对此类对象的动态进行建模，以及哪些表示适合学习和规划？那么，这些表示应该如何与规划和控制相结合，以期在机器人系统上实际实施操纵任务？第一个问题要求我们从定义适当的传感模式开始，以及对象表示-包括考虑基于分析或数值模拟的模型和基于数据驱动的机器学习的模型。在这项工作中，我们希望使用单目、立体和深度相机作为感知通道，以及各种基于结构化神经网络的架构来解释这些数据。图形界已经广泛地研究了布料状物体的模拟和可视化，尽管这些实现中的许多不适合实时计算。此外，许多这样的模型更多地是由高效可视化的目标驱动的，而不是现实的物理行为，部分原因是作为一种应对计算限制的手段。我们将研究这些模型的变化，以及机器学习方法的使用-这既是为了有效地实时模拟对象动力学，也是为了更好地根据观测数据校准模型，以期随后进行实时控制。这项研究还将使用这些模型来研究双手机器人的布料处理策略。这将包括基于模型的强化学习或模型预测控制体系结构的研究。这将包括研究新的集成感知、规划和控制架构，利用模型的结构特性来实现计算效率和行为的健壮性。本项目的实验将基于家庭和辅助用例。例如，机器人的行为包括叠衣服、铺床、叠被子和穿衣。在这个领域，挑战之一将是确保机器人能够安全地执行其任务，同时补偿人类代理可能执行的动作。因此，评价整个系统的指标包括任务完成情况和人机交互效率。我们设想在Baxter机器人上进行初步实验，以及一个使用UR10手臂和多指手的定制双手机器人平台。这便于多点抓取和动态操作，例如甩动、拖拽等。

Manipulating objects such as cloth (and ropes) is challenging for robots due to the complexity of the dynamics of these objects. For instance, most such objects will exhibit phenomena including wrinkles, folds, and crumpling or buckling - all of which are hard to model exactly, and difficult to predict in real time. This also causes perceptual problems, such as when parts of the cloth are obscured by itself, making it difficult to accurately observe and represent the current state of the object.Working in this domain, the core concerns of this research will include the following questions: How should we model the dynamics of such objects and what representations are tractable for learning and planning? Then, how should these representations be integrated with planning and control, with a view to practical implementation of manipulation tasks on robotic systems?The first question requires us to start from defining appropriate sensing modalities, along with object representations - including consideration of analytical or numerical simulation-based models and data-driven machine learning based models. In this work, we expect to use monocular, stereo and depth cameras as the perceptual modalities, and various structured neural network-based architectures for the interpretation of this data.The graphics community have extensively studied simulation and visualisation of cloth-like objects, although many of these implementations are not suited for real-time computation. Furthermore, many such models are driven more by objectives of efficient visualisation rather than realistic physical behaviour, in part as a means of coping with computational limitations. We will investigate variations of such models, alongside the use of machine learning methods - both for efficient re-al-time emulation of the object dynamics and for better calibration of models to observational data, with a view to subsequent real-time control.This research will also investigate cloth manipulation strategies for bi-manual robots, using such models. This will include investigation of model-based reinforcement learning or model-predictive control architectures. This will include investigation into new integrated perception, planning and control architectures that leverage structural properties of the models to achieve computational efficiency and robustness of behaviour.Experiments in this project will be based on domestic and assistive use cases. Example robot behaviours would be folding clothes, making a bed, tucking into bed and dressing. In this domain, one of the challenges will be to ensure that the robot can perform its task safely whilst compensating for the actions that a human agent may perform in turn. So, the metrics on which the overall system could be evaluated include task achievement and efficiency of human-robot interaction. We envision initial experiments on the Baxter robot, and a custom bi-manual robot platform using UR10 arms and multi-fingered hands. This facilitates multi-point grasping and dynamic manipulations such as flinging, tugging, etc.