Unsupervised Segmentation in Real-World Images via Spelke Object Inference
Unsupervised Segmentation in Real-World Images via Spelke Object Inference
复制标题
DOI：
10.48550/arxiv.2205.08515
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Honglin Chen;R. Venkatesh;Yoni Friedman;Jiajun Wu;J. Tenenbaum;Daniel L. K. Yamins;Daniel Bear
中科院分区：
文献类型：
--
作者：
Honglin Chen;R. Venkatesh;Yoni Friedman;Jiajun Wu;J. Tenenbaum;Daniel L. K. Yamins;Daniel Bear
自监督、类别无关的图像分割是计算机视觉领域的一个具有挑战性的开放性问题。在这里，我们展示了如何学习静态分组先验的运动自我监督的认知科学概念的Spelke对象：一组物理的东西，一起移动。我们介绍了兴奋抑制段提取网络（EISEN），它学习从基于运动的训练信号中提取静态场景的成对亲和图。EISEN然后使用一种新的图传播和竞争网络从亲和力产生片段。在训练过程中，经历相关运动的对象（如机器人手臂和它们移动的对象）通过自举过程解耦：EISEN解释了它已经学会分割的对象的运动。我们表明，EISEN在具有挑战性的合成和现实世界的机器人数据集上实现了最先进的自监督图像分割的实质性改进。
Self-supervised, category-agnostic segmentation of real-world images is a challenging open problem in computer vision. Here, we show how to learn static grouping priors from motion self-supervision by building on the cognitive science concept of a Spelke Object: a set of physical stuff that moves together. We introduce the Excitatory-Inhibitory Segment Extraction Network (EISEN), which learns to extract pairwise affinity graphs for static scenes from motion-based training signals. EISEN then produces segments from affinities using a novel graph propagation and competition network. During training, objects that undergo correlated motion (such as robot arms and the objects they move) are decoupled by a bootstrapping process: EISEN explains away the motion of objects it has already learned to segment. We show that EISEN achieves a substantial improvement in the state of the art for self-supervised image segmentation on challenging synthetic and real-world robotics datasets.