RUI: Ground-Truth Driven Stereo Algorithm Design

本项目旨在促进对基于能量的立体匹配技术的理解。这项研究使用新的具有地面实况差异的立体数据集，将解决这个问题：“什么是应该最小化的正确能量函数？”具体目标包括开发一种用于获得具有地面真实情况的高精度多视点立体数据集的自动化系统，并设计新的能量函数，其最小化产生的视差(1)准确地建模遮挡，(2)使用单目线索将遮挡边界与对象边界对齐，以及(3)在半遮挡区域创建合理的视差假设。在这些具有挑战性的任务中表现良好的立体算法在新兴的消费级应用中具有重要的应用，包括用于电话会议的虚拟凝视校正，以及用于3D可视化的基于图像的对象模型的创建。这项研究还将专注于开发快速、近似的能量最小化技术，包括图割和动态规划方法的变体。本科生将积极参与这项研究的所有组成部分，特别是在数据采集和测试阶段。该项目的智力价值是为计算机视觉社区提供高质量的、具有基本事实的多视角数据集，并提高基于能量的图像匹配技术的最新水平。该项目的更广泛影响包括提高了计算机视觉方法在日益成为社会核心的应用程序中的适用性，如电信和电子商务；以及有机会让文科大学的本科生接触到研究、实验和发现的世界。

This project aims to advance the understanding of energy-based stereo-matching techniques. Using new stereo data sets with ground-truth disparities, the research will address the question: "What is the right energy function to minimize?" The specific goals include developing an automated system for obtaining highly accurate multi-view stereo data sets with ground truth, and designing new energy functions whose minimization yields disparities that (1) accurately model occlusion, (2) align occlusion boundaries with object boundaries using monocular cues, and (3) create reasonable disparity hypotheses in half-occluded regions. Stereo algorithms that perform well on these challenging tasks have important applications in emerging consumer-level applications, including virtual gaze correction for teleconferencing, and the creation of image-based object models for 3D visualization. The research will also focus on developing fast, approximate energy minimization techniques, including variants of graph-cut and dynamic programming methods. Undergraduate students will be actively involved in all components of this research, in particular in the data acquisition and testing stages. The intellectual merits of the project are to provide the computer vision community with high-quality, multi-view data sets with ground truth and to improve the state of the art in energy-based image-matching techniques. The broader impacts of the project include the improved applicability of computer vision methods to applications that are becoming central to society, such as telecommunication and e-commerce; and the opportunity to expose undergraduates at a liberal-arts college to the world of research, experimentation, and discovery.