CAREER: Optimizing Computational Range and Velocity Imaging

该项目专注于为新兴的计算距离和速度成像开发优化的硬件和软件实现。今天，获取距离和速度的主要技术是雷达和激光雷达。这些系统提供了非常高的精度，但现有的系统昂贵、笨重且速度慢，因为它们以逐点的方式顺序扫描场景。飞行时间(ToF)相机已经成为廉价而快速的替代品。TOF相机使用主动的时间调制照明和编码的像素内传感来估计相机与三维(3D)中每个场景点之间的距离。最近，TOF相机首次展示了同步距离和速度成像技术。要充分发挥距离和速度成像的变革性潜力，一个主要障碍是商用TOF相机的低电平传感器和照明功能的使用受限。距离(或深度)和速度成像使计算机能够感知和理解世界和3D场景动态。医学成像、国防、人机交互和机器人等领域的广泛应用依赖于深度和速度信息来执行特定领域的任务，如目标检测、跟踪、定位、测绘和运动分析。该研究通过优化新兴计算成像系统在直接和非视距场景中的速度、分辨率、深度和速度估计的精度、3D成像能力和光子灵敏度，使计算范围和速度成像变得实用。通过分析时间分辨成像系统的基本局限性和优点，设计了优化的硬件实现和重建算法，以促进新的距离和速度传感能力并使其实用(即健壮、廉价和可重复性)。这些预期的见解和贡献促进了对时间分辨计算成像的局限性以及如何实际实现它们的认识和理解。开发的计算成像系统和数学模型有望为计算机和机器视觉、医学成像、显微镜、科学成像、遥感、国防和机器人等领域的各种应用提供全新的基础。

This project focuses on developing optimized hardware and software implementations for emerging computational range and velocity imaging. Today, the primary technologies for capturing range and velocity are radar and lidar. These offer a very high precision, but available systems are expensive, bulky, and slow, because they sequentially scan scenes in a point-by-point manner. Time-of-flight (ToF) cameras have emerged as inexpensive and fast alternatives. ToF cameras use active, temporally-modulated illumination and coded, in-pixel sensing to estimate the distance between the camera and each scene point in three dimensions (3D). Recently, simultaneous range and velocity imaging techniques were demonstrated for the first time with ToF cameras. A major roadblock for unlocking the full, transformative potential of range and velocity imaging has been the limited access to low-level sensor and illumination functionalities of commercially-available ToF cameras. Range (or depth) and velocity imaging enables computers to sense and understand the world and 3D scene dynamics. A wide range of applications in medical imaging, defense, human-computer interaction, and robotics rely on depth and velocity information to perform domain-specific tasks, such as object detection, tracking, localization, mapping, and motion analysis.This research makes computational range and velocity imaging practical by optimizing the speed, resolution, precision of depth and velocity estimation, 3D imaging capabilities, and photon sensitivity of emerging computational imaging systems in direct and non-line-of-sight scenarios. By analyzing the fundamental limitations and benefits of time-resolved imaging systems, optimized hardware implementations and reconstruction algorithms are devised that facilitate novel range and velocity sensing capabilities and make them practical (i.e. robust, inexpensive, and reproducible). The anticipated insights and contributions advance knowledge and gain an understanding of the limits of time-resolved computational imaging and how to practically achieve them. The developed computational imaging systems and mathematical models are expected to provide fundamentally new building blocks for a diversity of applications in computer and machine vision, medical imaging, microscopy, scientific imaging, remote sensing, defense, and robotics.