Learning Visual Shape Control of Novel 3D Deformable Objects from Partial-View Point Clouds
Learning Visual Shape Control of Novel 3D Deformable Objects from Partial-View Point Clouds
复制标题
DOI：
10.1109/icra46639.2022.9812215
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Bao Thach;Brian Y. Cho;A. Kuntz;Tucker Hermans
中科院分区：
文献类型：
--
作者：
Bao Thach;Brian Y. Cho;A. Kuntz;Tucker Hermans
如果机器人能够可靠地操纵3D可变形物体的形状，它们就可以在从家庭护理到仓库配送到手术辅助等领域找到应用。弹性三维可变形物体的解析模型需要许多参数来描述确定物体形状时存在的潜在无限自由度。以前在执行3D形状控制方面的尝试依赖于手工制作的特征来表示对象形状，并且需要训练对象特定的控制模型。我们通过使用我们新颖的DeformerNet神经网络架构来克服这些问题，该架构在被操作对象的局部视图点云和目标形状的点云上操作，以学习对象形状的低维表示。这种形状嵌入使机器人能够学习定义一个视觉伺服控制器，该控制器为机器人末端执行器提供笛卡尔姿态变化，从而使物体朝着目标形状变形。至关重要的是，我们在模拟和物理机器人上都证明了DeformerNet可靠地概括了训练期间未见的物体形状和材料刚度，并且在通用形状控制和手术回缩任务方面优于比较方法。
If robots could reliably manipulate the shape of 3D deformable objects, they could find applications in fields ranging from home care to warehouse fulfillment to surgical assistance. Analytic models of elastic, 3D deformable objects require numerous parameters to describe the potentially infinite degrees of freedom present in determining the object's shape. Previous attempts at performing 3D shape control rely on hand-crafted features to represent the object shape and require training of object-specific control models. We overcome these issues through the use of our novel DeformerNet neural network architecture, which operates on a partial-view point cloud of the object being manipulated and a point cloud of the goal shape to learn a low-dimensional representation of the object shape. This shape embedding enables the robot to learn to define a visual servo controller that provides Cartesian pose changes to the robot end-effector causing the object to deform towards its target shape. Crucially, we demonstrate both in simulation and on a physical robot that DeformerNet reliably generalizes to object shapes and material stiffness not seen during training and outperforms comparison methods for both the generic shape control and the surgical task of retraction.