Seeing Through your Skin: Recognizing Objects with a Novel Visuotactile Sensor
Seeing Through your Skin: Recognizing Objects with a Novel Visuotactile Sensor
复制标题
透视你的皮肤：用新型视觉触觉传感器识别物体
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
G. Dudek
中科院分区：
文献类型：
--
作者：
F. Hogan;M. Jenkin;S. Rezaei;Yogesh A. Girdhar;D. Meger;G. Dudek
我们介绍了一类新的基于视觉的传感器和相关的算法过程，结合联合收割机视觉成像与高分辨率的触觉发送，所有在一个统一的硬件和计算架构。我们证明了该传感器在多模态物体识别和计量方面的功效。对象识别通常被制定为一个单峰的任务，但通过结合两个传感器的方式，我们表明，我们可以实现几个显着的性能改进。该传感器名为See-Through-your-Skin传感器（STS），旨在提供接触表面的丰富多模式传感。受光学触觉传感技术最新发展的启发，我们解决了这些传感器的一个关键缺失功能：捕捉接触表面以外区域的视觉透视的能力。而光学触觉传感器通常是不透明的，我们提出了一个传感器与一个透明的皮肤，具有双重功能的触觉传感器和/或作为视觉相机，这取决于其内部的照明条件。本文详细介绍了传感器的设计，展示了其双重传感功能，并提出了一种融合视觉和触摸的深度学习架构。我们验证了传感器的能力，以分类家居用品，识别精细纹理，并推断其物理特性都通过数值模拟和实验与智能台面原型。
We introduce a new class of vision-based sensor and associated algorithmic processes that combine visual imaging with high-resolution tactile sending, all in a uniform hardware and computational architecture. We demonstrate the sensor’s efficacy for both multi-modal object recognition and metrology. Object recognition is typically formulated as an unimodal task, but by combining two sensor modalities we show that we can achieve several significant performance improvements. This sensor, named the See-Through-your-Skin sensor (STS), is designed to provide rich multi-modal sensing of contact surfaces. Inspired by recent developments in optical tactile sensing technology, we address a key missing feature of these sensors: the ability to capture a visual perspective of the region beyond the contact surface. Whereas optical tactile sensors are typically opaque, we present a sensor with a semitransparent skin that has the dual capabilities of acting as a tactile sensor and/or as a visual camera depending on its internal lighting conditions. This paper details the design of the sensor, showcases its dual sensing capabilities, and presents a deep learning architecture that fuses vision and touch. We validate the ability of the sensor to classify household objects, recognize fine textures, and infer their physical properties both through numerical simulations and experiments with a smart countertop prototype.