RI: Small: Learning to Eliminate Heuristics in Stereo Vision

该项目开发的技术，以提高立体和多视图立体算法通过消除启发式和手动调整使用机器学习技术。立体匹配是估计场景中点的深度或3D坐标的过程，通过估计两个或多个图像中像素或其他原语之间的对应关系来实现。然而，即使是目前最成功的立体匹配算法，也使用了大量的启发式算法。本项目开发的方法消除了双目和多视角立体匹配中的启发式算法，提供了更高的精度、结果的可解释性和更高的可移植性。立体视觉在3D建模、增强现实、驾驶辅助、自主导航和人机交互等许多应用中发挥着重要作用。该项目的教育和推广方面侧重于让K-12和本科生参与STEM教育和研究。这项研究通过训练分类器来解决立体视觉问题，这些分类器从成对或更大的图像集中学习，具有地面真值深度，可以对未观察到的数据做出比手工规则获得的数据更准确的预测。该方法是全面的，解决了双目立体匹配过程的所有阶段，包括匹配成本函数，成本聚合，优化和细化。该框架还支持基于表面斑块、深度图或占用网格的多视图立体表示及其相应的算法。随机森林分类器非常适合在非均匀特征空间中使用，分类器校准可以确保它们的输出接近所考虑的类的真实后验概率。由此产生的算法和发现可以转移到其他需要像素对应的计算机视觉问题，如光流估计、图像拼接和模板匹配。

This project develops technologies to improve stereo and multi-view stereo algorithms by removing heuristics and hand-tuning using machine learning techniques. Stereo matching is the process of estimating depth of points, or 3D coordinates in a scene, and is enabled by the estimation of correspondences between pixels or other primitives in two or more images. Even the most successful current stereo matching algorithms, however, use a large number of heuristics. The developed methods from this project eliminate the heuristics from binocular and multi-view stereo matching and deliver algorithms with higher accuracy, interpretability of the results and higher portability to different settings. Stereo vision plays an important role in many applications, such as 3D modeling, augmented reality, driver assistance, autonomous navigation and human computer interaction. The educational and outreach aspects of the project focus on involving K-12 and undergraduate students in STEM education and research. This research addresses stereo vision by training classifiers that learn from pairs, or larger sets of images, with ground truth depth to make more accurate predictions about unobserved data than those obtained by hand-crafted rules. The approach is comprehensive and tackles all stages of the binocular stereo matching process, including the matching cost function, cost aggregation, optimization and refinement. Representations for multi-view stereo based on surface patches, depth maps or occupancy grids and the corresponding algorithms are also supported by the same framework. Random forest classifiers are well suited for use in inhomogeneous feature spaces and classifier calibration can ensure that their outputs are close to the true posterior probabilities of the classes under consideration. The resulting algorithms and findings can be transferred to other computer vision problems that require pixel correspondences, such as optical flow estimation, image stitching and template matching.