Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation
Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation
复制标题
DOI：
10.48550/arxiv.2206.02881
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Zixuan Huang;Xingyu Lin;David Held
中科院分区：
文献类型：
--
作者：
Zixuan Huang;Xingyu Lin;David Held
自遮挡对于布料操作来说是具有挑战性的，因为它使得难以估计布料的完整状态。理想情况下，一个试图展开皱巴巴或折叠的布料的机器人应该能够对布料的被遮挡区域进行推理。我们利用布料姿态估计的最新进展来构建一个系统，该系统使用显式的遮挡推理来展开皱巴巴的布料。具体来说，我们首先学习一个模型来重建布料的网格。然而，由于布料配置的复杂性以及遮挡造成的模糊性，该模型可能会有误差。我们的主要见解是，我们可以通过使用自监督损失进行测试时微调来进一步细化预测的重建结果。所获得的重建网格使我们能够在对遮挡进行推理的同时使用基于网格的动力学模型进行规划。我们在布料展平和布料规范化（其目标是将布料操作成规范姿态）方面对我们的系统进行了评估。我们的实验表明，我们的方法明显优于那些没有明确考虑遮挡或不进行测试时优化的先前方法。视频和可视化内容可以在我们的项目网站（https://sites.google.com/view/occlusion-reason/home）上找到。
Self-occlusion is challenging for cloth manipulation, as it makes it difficult to estimate the full state of the cloth. Ideally, a robot trying to unfold a crumpled or folded cloth should be able to reason about the cloth's occluded regions. We leverage recent advances in pose estimation for cloth to build a system that uses explicit occlusion reasoning to unfold a crumpled cloth. Specifically, we first learn a model to reconstruct the mesh of the cloth. However, the model will likely have errors due to the complexities of the cloth configurations and due to ambiguities from occlusions. Our main insight is that we can further refine the predicted reconstruction by performing test-time finetuning with self-supervised losses. The obtained reconstructed mesh allows us to use a mesh-based dynamics model for planning while reasoning about occlusions. We evaluate our system both on cloth flattening as well as on cloth canonicalization, in which the objective is to manipulate the cloth into a canonical pose. Our experiments show that our method significantly outperforms prior methods that do not explicitly account for occlusions or perform test-time optimization. Videos and visualizations can be found on our $\href{https://sites.google.com/view/occlusion-reason/home}{\text{project website}}.$