RI: Small: RUI: Benchmarks and Algorithms for Mobile Image Matching

这个项目将为移动的图像匹配领域提供新的基准和新的算法。 为了推动和关注移动的图像匹配的新研究，现有的米德尔伯里基准将增加新的数据集的校准多视图和视频序列采集与移动的设备，连同地面真理几何。该项目还将为稳健和可扩展的图像匹配提供新的算法方法。本科生将积极参与该项目的所有组成部分，特别是在数据采集和测试阶段，以及在线评估工具的创作。该项目将产生多方面的影响。 首先，具有挑战性的基准将成为新研究的催化剂。 除了基准测试之外，高质量的数据集也很有用，因为它们可以帮助算法设计并实现学习方法。 其次，为移动的设备量身定制的可扩展和强大的匹配技术将使大量新的应用程序对广大人群产生广泛的影响，包括用于社交媒体的对象和人的交互式3D建模，在线商务以及增强和虚拟现实。 最后，该项目将使佛蒙特州一所文科大学的本科生接触到研究、实验和发现的世界。该项目将通过校准的多视图和视频序列来增强现有的米德尔伯里数据集，这些序列是通过机器人手臂上的移动的设备获得的，具有挑战性的场景具有已知的几何形状，使用结构化照明获得。 数据集将包括IMU数据和闪光/无闪光图像对。 该项目还将探索新的评估指标以及高质量合成图像序列的利用。 新数据集的一个子集将用于移动的3D重建任务的新基准。该算法的工作将有助于新的方法，鲁棒性和可扩展的图像匹配。 虽然社区目前的趋势是从大量标记的训练数据中学习通用模型，但该项目的目标是在匹配过程中从手头的图像中学习数据术语。 这种自我调整的数据项将模拟辐射和几何失真，而不是对它们保持不变。 另一个重点将是内存效率的方法，避免了一个完整的匹配空间的穷举搜索，同时显式地推理遮挡，反射和透明度。 其他算法技术将包括基于层的图像匹配算法，适用于快速和可扩展的图像匹配的新平滑度术语，以及处理完全无纹理场景的新策略。

This project will provide both new benchmarks and new algorithms for the mobile image matching domain. In order to drive and focus new research on mobile image matching, the existing Middlebury benchmarks will be augmented with new datasets of calibrated multiview and video sequences acquired with mobile devices, together with ground-truth geometry. The project will also contribute novel algorithmic approaches for robust and scalable image matching. Undergraduate students will be actively involved in all components of this project, in particular in the data acquisition and testing stages, as well as the authoring of online evaluation tools. The project will have several broader impacts. First, challenging benchmarks will serve as catalysts for new research. High-quality datasets are also useful beyond benchmarking in that they can aid algorithm design and enable learning approaches. Second, scalable and robust matching techniques tailored for mobile devices will enable a host of new applications with broad impacts on the population at large, including interactive 3D modeling of objects and people for social media, online commerce, and augmented and virtual reality. Finally, the project will expose undergraduates at a liberal-arts college in rural Vermont to the world of research, experimentation, and discovery.This project will augment the existing Middlebury datasets with calibrated multi-view and video sequences acquired with mobile devices from a robot arm, of challenging scenes with known geometry, derived using structured lighting. Datasets will include IMU data and flash/no-flash image pairs. The project will also explore novel evaluation metrics as well as the utilization of high-quality synthetic image sequences. A subset of the new datasets will be employed in new benchmarks for mobile 3D reconstructions tasks. The algorithmic work will contribute novel approaches for robust and scalable image matching. While the current trend in the community is to learn general models from large sets of labeled training data, this project will instead aim to learn data terms from the images at hand during the matching process. Such self-adjusting data terms will model radiometric and geometric distortions rather than being invariant to them. Another focus will be on memory-efficient approaches that avoid an exhaustive search of the full matching space while explicitly reasoning about occlusion, reflections, and transparency. Additional algorithmic techniques will include layer-based image matching algorithms, novel smoothness terms suitable for fast and scalable image matching, and novel strategies for dealing with completely textureless scenes.