Learning to reason from uncalibrated wide angle images

我们看到，在许多现实应用中，包括安全、增强现实（AR）、医疗保健和自主系统，对使用宽视场（FOV）镜头的摄像机的需求越来越大。增加的视野使成本、能量和计算最小化，因为需要的相机更少。使用这种透镜的缺点是，它们需要使用透镜校准方法来获得透镜畸变轮廓，然后使用该轮廓来校正图像-即取消畸变的影响并创建透视投影图像。已经提出了一系列这样的方法，从经典（确定性）到基于深度学习。不幸的是，由于许多预处理和后处理步骤，校准在大多数关键系统中被认为是一种负担。在这个提议中，我们建议打破“校准和校正”的范式，而是直接在广角图像上进行推理。因此，该项目的目标是开发一套理论和实验工具，用于对未经校准的广角镜头拍摄的图像进行场景理解。

We witness an increasing demand of using cameras with wide field of view (FOV) lenses in many real-life applications, including security, augmented reality (AR), healthcare and autonomous systems. The increased field of view minimizes cost, energy and computation since fewer cameras are needed. The downside of using such lenses is that they require the use of lens calibration methods to obtain a lens distortion profile, which is then used to rectify the image-that is, to cancel the effects of distortion and create a perspective projection image. A wide array of such methods, ranging from classical (deterministic) to deep-learning-based have been proposed. Unfortunately, calibration is considered as a burden in most critical systems due to the many pre- and post-processing steps.In this proposal, we propose to break free from the "calibrate-and-rectify" paradigm, and instead directly reason on the wide-angle images. The goal of this project is therefore to develop a set of theoretical and experimental tools for scene understanding on images captured with uncalibrated, wide-angle lenses.