RI: Small: Collaborative Research: Structured Inference for Low-Level Vision

视觉是一种有价值的感知方式，因为它是通用的。它可以让人类在不熟悉的环境中导航，发现资产，掌握和操纵工具，对投射物做出反应，通过混乱跟踪目标，解释肢体语言，并识别熟悉的物体和人。这种多功能性源于低水平的视觉过程，从模糊的视网膜测量中，产生了有用的深度，表面取向，运动和其他内在场景属性的中间表示。该项目为机器中类似的低级处理建立了数学和计算基础。它解决的关键挑战是如何有效地编码和利用这样一个事实，即视觉上，世界表现出实质性的内在结构。本项目通过加深对机器低级视觉的理解，使计算机视觉系统在准确性、可靠性、速度和功率效率方面与人类视觉相媲美。本研究重新审视低级视觉，并开发一个综合框架，该框架对来自不同光学线索的信息具有共同的抽象;跨大区域和多尺度编码场景结构的能力;并行和分布式处理的实现;以及大规模的端到端可学习性。该项目将低层次视觉作为一个结构化的预测任务，将来自许多重叠感受野的模糊局部预测结合起来，以产生一个跨越视野的一致的全局场景图。结构化预测模型与用于语义分割等分类任务的模型不同，因为它们专门设计用于适应低级视觉的独特要求和属性：连续值输出空间;可能形成等概率流形的模糊性;极端尺度变化;以及具有高阶分段平滑度的全局场景映射。通过加强低级视觉的计算基础，该项目致力于实现更高效、更通用的多种视觉系统，并努力在计算机视觉领域产生影响。

Vision is a valuable sensing modality because it is versatile. It lets humans navigate through unfamiliar environments, discover assets, grasp and manipulate tools, react to projectiles, track targets through clutter, interpret body language, and recognize familiar objects and people. This versatility stems from low-level visual processes that somehow produce, from ambiguous retinal measurements, useful intermediate representations of depth, surface orientation, motion, and other intrinsic scene properties. This project establishes a mathematical and computational foundation for similar low-level processing in machines. The key challenge it addresses is how to usefully encode and exploit the fact that, visually, the world exhibits substantial intrinsic structure. By advancing understanding of low-level vision in machines, this project makes progress toward computer vision systems that can compare to vision in humans, in terms of accuracy, reliability, speed, and power-efficiency.This research revisits low-level vision, and develops a comprehensive framework that possesses a common abstraction for information from different optical cues; the ability to encode scene structure across large regions and at multiple scales; implementation as parallel and distributed processing; and large-scale end-to-end learnability. The project approaches low-level vision as a structured prediction task, with ambiguous local predictions from many overlapping receptive fields being combined to produce a consistent global scene map that spans the visual field. The structured prediction models are different from those used for categorical tasks such as semantic segmentation, because they are specifically designed to accommodate the distinctive requirements and properties of low-level vision: continuous-valued output spaces; ambiguities that may form equiprobable manifolds; extreme scale variations; and global scene maps with higher-order piecewise smoothness. By strengthening the computational foundations of low-level vision, this project strives to enable many kinds of vision systems that are more efficient and more versatile, and it strives to have impacts across the breadth of computer vision.