ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer
ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer
复制标题
DOI：
10.1109/cvpr52688.2022.01034
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
Ruohan Gao;Zilin Si;Yen-Yu Chang;Samuel Clarke;J. Bohg;Li Fei-Fei-Li-Fei-Fei-48004138;Wenzhen Yuan;Jiajun Wu
中科院分区：
文献类型：
--
作者：
Ruohan Gao;Zilin Si;Yen-Yu Chang;Samuel Clarke;J. Bohg;Li Fei-Fei-Li-Fei-Fei-48004138;Wenzhen Yuan;Jiajun Wu
物体在我们的日常活动中起着至关重要的作用。尽管以物体为中心的多感官学习最近显示出巨大潜力，但先前工作中对物体的建模相当不切实际。ObjectFolder 1.0是一个近期的数据集，它引入了100个具有视觉、听觉和触觉感官数据的虚拟物体。然而，该数据集规模较小，多感官数据质量有限，阻碍了对现实场景的泛化。我们提出了ObjectFolder 2.0，这是一个大规模的常见家居物体多感官数据集，采用隐式神经表示形式，在三个方面显著增强了ObjectFolder 1.0。首先，我们的数据集在物体数量上大10倍，渲染时间快几个数量级。其次，我们显著提高了所有三种模态的多感官渲染质量。第三，我们表明从我们数据集中的虚拟物体学习到的模型在三个具有挑战性的任务中成功地迁移到它们在现实世界中的对应物上：物体尺度估计、接触定位和形状重建。ObjectFolder 2.0为计算机视觉和机器人技术中的多感官学习提供了一条新路径和测试平台。该数据集可在https://github.com/rhgao/ObjectFolder获取。
Objects play a crucial role in our everyday activities. Though multisensory object-centric learning has shown great potential lately, the modeling of objects in prior work is rather unrealistic. ObjectFolder 1.0 is a recent dataset that introduces 100 virtualized objects with visual, acoustic, and tactile sensory data. However, the dataset is small in scale and the multisensory data is of limited quality, hampering generalization to real-world scenarios. We present ObjectFolder 2.0, a large-scale, multisensory dataset of common household objects in the form of implicit neural representations that significantly enhances ObjectFolder 1.0 in three aspects. First, our dataset is 10 times larger in the amount of objects and orders of magnitude faster in rendering time. Second, we significantly improve the multisensory rendering quality for all three modalities. Third, we show that models learned from virtual objects in our dataset successfully transfer to their real-world counterparts in three challenging tasks: object scale estimation, contact localization, and shape reconstruction. ObjectFolder 2.0 offers a new path and testbed for multisensory learning in computer vision and robotics. The dataset is available at https://github.com/rhgao/ObjectFolder.