Teaching Cameras to Feel: Estimating Tactile Physical Properties of Surfaces From Images
Teaching Cameras to Feel: Estimating Tactile Physical Properties of Surfaces From Images
复制标题
DOI：
10.1007/978-3-030-58583-9_1
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Matthew Purri;Kristin J. Dana
中科院分区：
文献类型：
--
作者：
Matthew Purri;Kristin J. Dana
视觉输入和触觉感知之间的联系对于物体操作任务（如抓握和推动）至关重要。在这项工作中，我们介绍了具有挑战性的任务，估计一组触觉物理特性的视觉信息。我们的目标是建立一个模型，学习视觉信息和触觉物理属性之间的复杂映射。我们构建了第一个具有400多个多视图图像序列和相应触觉属性的图像触觉数据集。共十五个触觉物理性质的类别，包括摩擦，顺应性，附着力，纹理和热导测量，然后估计我们的模型。我们开发了一个跨模态框架，包括一个对抗性的目标和一个新的视觉-触觉联合分类损失。此外，我们引入了一个神经架构搜索框架，能够选择最佳的视角组合，以估计给定的物理特性。
The connection between visual input and tactile sensing is critical for object manipulation tasks such as grasping and pushing. In this work, we introduce the challenging task of estimating a set of tactile physical properties from visual information. We aim to build a model that learns the complex mapping between visual information and tactile physical properties. We construct a first of its kind image-tactile dataset with over 400 multiview image sequences and the corresponding tactile properties. A total of fifteen tactile physical properties across categories including friction, compliance, adhesion, texture, and thermal conductance are measured and then estimated by our models. We develop a cross-modal framework comprised of an adversarial objective and a novel visuo-tactile joint classification loss. Additionally, we introduce a neural architecture search framework capable of selecting optimal combinations of viewing angles for estimating a given physical property.