RI: Small: Collaborative Research: Structured Inference for Low-Level Vision

视觉是一种有价值的感知方式，因为它是通用的。它让人类在不熟悉的环境中导航，发现资产，掌握和操纵工具，对投射物做出反应，在混乱中跟踪目标，解释肢体语言，识别熟悉的物体和人。这种多功能性源于低级视觉过程，这些过程以某种方式从模糊的视网膜测量中产生有用的深度，表面方向，运动和其他固有场景属性的中间表示。该项目为类似的机器低级处理建立了数学和计算基础。它解决的关键挑战是如何有效地编码和利用这样一个事实，即从视觉上看，世界呈现出实质性的内在结构。通过提高对机器低级视觉的理解，该项目在计算机视觉系统方面取得了进展，在准确性、可靠性、速度和能效方面可以与人类视觉相媲美。本研究重新审视了低层次视觉，并开发了一个综合框架，该框架对来自不同光学线索的信息具有共同的抽象；在大区域和多尺度上对场景结构进行编码的能力；实现为并行和分布式处理；大规模的端到端可学习性。该项目将低级视觉作为一个结构化的预测任务，将来自许多重叠的接受域的模糊局部预测结合起来，产生跨越视野的一致的全局场景地图。结构化预测模型不同于用于分类任务（如语义分割）的预测模型，因为它们是专门为适应低层次视觉的独特要求和特性而设计的：连续值输出空间；可能形成等概率流形的模糊性；极端尺度变化；以及具有高阶分段平滑的全局场景地图。通过加强低级视觉的计算基础，该项目努力使多种视觉系统更高效、更通用，并努力在计算机视觉的广度上产生影响。

Vision is a valuable sensing modality because it is versatile. It lets humans navigate through unfamiliar environments, discover assets, grasp and manipulate tools, react to projectiles, track targets through clutter, interpret body language, and recognize familiar objects and people. This versatility stems from low-level visual processes that somehow produce, from ambiguous retinal measurements, useful intermediate representations of depth, surface orientation, motion, and other intrinsic scene properties. This project establishes a mathematical and computational foundation for similar low-level processing in machines. The key challenge it addresses is how to usefully encode and exploit the fact that, visually, the world exhibits substantial intrinsic structure. By advancing understanding of low-level vision in machines, this project makes progress toward computer vision systems that can compare to vision in humans, in terms of accuracy, reliability, speed, and power-efficiency.This research revisits low-level vision, and develops a comprehensive framework that possesses a common abstraction for information from different optical cues; the ability to encode scene structure across large regions and at multiple scales; implementation as parallel and distributed processing; and large-scale end-to-end learnability. The project approaches low-level vision as a structured prediction task, with ambiguous local predictions from many overlapping receptive fields being combined to produce a consistent global scene map that spans the visual field. The structured prediction models are different from those used for categorical tasks such as semantic segmentation, because they are specifically designed to accommodate the distinctive requirements and properties of low-level vision: continuous-valued output spaces; ambiguities that may form equiprobable manifolds; extreme scale variations; and global scene maps with higher-order piecewise smoothness. By strengthening the computational foundations of low-level vision, this project strives to enable many kinds of vision systems that are more efficient and more versatile, and it strives to have impacts across the breadth of computer vision.