RI: Small: Depth from Differential Defocus

该项目将探索一种新的深度传感器。新的传感器通过单个透镜观察光学散焦的微小变化来操作，并且它们需要非常少量的数字计算。这些传感器的显着特点是，它们可以比现有的深度传感器技术更小，功耗更低。通过在较小的平台上实现深度传感功能，它们有助于加速创建智能微型系统和有效的物联网。深度传感器产生二维图像，其中每个像素的值是沿对应射线沿着到场景点的距离。这些传感器种类繁多，它们已经推动了自主导航、手势驱动界面、机器人等领域的进步。这项研究将开发基于一种新的视觉提示（称为差分散焦）的传感器。就像众所周知的立体声和散焦深度的被动深度提示一样，这种新的提示避免了在广播光上花费功率。但与现有的被动提示不同，它使用易于计算的简单解析表达式来计算深度。为了建立差分散焦作为一种新的深度感测方法，该项目旨在发现一个完整的知识堆栈，从数学基础到算法和硬件原型。数学基础包括对应于许多形式的差分散焦的深度约束的目录，诸如差分相机运动、传感器运动、焦距的改变和光圈的改变。在硬件层面，该项目将追求单镜头和多镜头设计，包括可变形透镜和定制的光电传感器。在数学上，该项目将探索使用反向传播来微调深度计算参数的方法。更进一步，它将探索反向传播到光学维度，以便以协同的方式一起优化光学和计算参数。

This project will explore a new class of depth sensors. The new sensors operate by observing small changes in optical defocus through a single lens, and they require very small amounts of digital computation. The distinguishing feature of these sensors is that they can be much smaller and lower power than existing depth sensor technologies. By enabling depth sensing capabilities on smaller platforms, they help accelerate the creation of smart micro-scale systems and an effective Internet of things. Depth sensors produce two-dimensional images where each pixel's value is the distance to a scene point along a corresponding ray. A variety of these sensors exist, and they are already fueling advances in autonomous navigation, gesture-driven interfaces, robotics, and more.This research will develop sensors based on a new visual cue called differential defocus. Like the well-known passive depth cues of stereo and depth-from-defocus, this new cue avoids spending power on broadcasting light. But unlike the existing passive cues, it calculates depth using simple analytic expressions that are easy to compute. To establish differential defocus as a new way to sense depth, this project aims to discover a complete stack of knowledge, from mathematical foundations to algorithms and hardware prototypes. The mathematical foundations include a catalog of depth constraints that correspond to many forms of differential defocus, such as differential camera motion, sensor motion, change of focal length, and change of aperture. At the hardware level, the project will pursue both single-shot and multi-shot designs that incorporate deformable lenses and customized photosensors. Algorithmically, the project will explore methods for using back-propagation to fine-tune the parameters of depth computations. Going further, it will explore back-propagation into the optical dimension, in order to enable the optimization of optical and computational parameters together, in a synergistic manner.