ASIC: Aligning Sparse in-the-wild Image Collections
ASIC: Aligning Sparse in-the-wild Image Collections
复制标题
DOI：
10.1109/iccv51070.2023.00382
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
Kamal Gupta;V. Jampani;Carlos Esteves;Abhinav Shrivastava;A. Makadia;Noah Snavely;Abhishek Kar
中科院分区：
文献类型：
--
作者：
Kamal Gupta;V. Jampani;Carlos Esteves;Abhinav Shrivastava;A. Makadia;Noah Snavely;Abhishek Kar
提出了一种对象类别稀疏野外图像集合联合对齐的方法。大多数以前的工作假设要么是基本真实的关键点注释，要么是单个对象类别的图像的大型数据集。然而，对于世界上存在的物体的长尾来说，上述假设都不成立。我们提出了一种自监督技术，它直接对特定对象/对象类别的稀疏图像集合进行优化，以获得整个集合上一致的密集对应。我们使用从预先训练的视觉变换(VIT)模型的深层特征获得的成对最近邻作为噪声和稀疏关键点匹配，并通过优化神经网络将图像集合联合映射到学习的规范网格来使其密集而准确地匹配。在CUB、SPAIR-71K和PF-Willow基准测试上的实验表明，与现有的自监督方法相比，我们的方法可以在整个图像集合中产生全局一致和更高质量的对应。代码和其他材料将在https://kampta.github.io/asic.上提供
We present a method for joint alignment of sparse in-the-wild image collections of an object category. Most prior works assume either ground-truth keypoint annotations or a large dataset of images of a single object category. However, neither of the above assumptions hold true for the long-tail of the objects present in the world. We present a self-supervised technique that directly optimizes on a sparse collection of images of a particular object/object category to obtain consistent dense correspondences across the collection. We use pairwise nearest neighbors obtained from deep features of a pre-trained vision transformer (ViT) model as noisy and sparse keypoint matches and make them dense and accurate matches by optimizing a neural network that jointly maps the image collection into a learned canonical grid. Experiments on CUB, SPair-71k and PF-Willow benchmarks demonstrate that our method can produce globally consistent and higher quality correspondences across the image collection when compared to existing self-supervised methods. Code and other material will be made available at https://kampta.github.io/asic.