Computational and Optical Processing Architectures for next-Generation Mobile Cameras

移动的设备正在推动我们与他人以及与周围世界互动方式的重大转变。它们准备成为我们自身的延伸，具有大幅增加的感知物理世界的能力。相机是移动的设备的“眼睛”，其光学质量、速度和无线联网能力的不断改进催生了新的市场和新的用途（例如，基于图像的搜索，以及为视障人士提供的辅助技术）。 虽然过去在移动的相机方面的工作主要集中在让它们“看得更好”（更精细的细节，每秒更多的图像，需要更少的光线），但现在是时候问“移动的相机还能看到什么”了。拟议的工作将开发和实际演示可能的应用程序与一个完全不同的相机，从来没有见过的能力。我们的相机技术不仅能以三维方式（即，我们的相机技术基于PI开创的一种称为Primal-Dual Coding的新型成像技术，可实现3D扫描、视觉识别、生物识别、检测和科学成像等众多应用。我们已经证明了使用现成组件的潜在工作原理的潜力。在这个项目中，我们试图通过（1）制造定制的CMOS成像器，（2）设计和构建新的移动的相机平台，以及（3）开发相关的硬件和软件架构来支持它们，从而释放我们相机技术的真正潜力。

Mobile devices are fuelling a major shift in how we interact with each other and with the world around us. They are poised to become an extension of ourselves, with drastically-increased capabilities for sensing the physical world. Cameras are the "eyes" of mobile devices, and continuous improvements in their optical quality, speed and wireless networking capabilities have spawned new markets and new uses (e.g., image-based search, and assistive technologies for the visually impaired). While past work in mobile cameras focused primarily on making them "see better" (finer detail, more images per second, requiring less light), it is now time to ask "what else can mobile cameras see". The proposed work will develop and practically demonstrate the applications possible with a radically-different camera that has never-seen-before abilities. Not only will our camera technology image the world in three dimensions (i.e., the distance and 3D shape of objects it "sees") at a detail that far surpasses existing solutions, but it can take into account the 3D path that light followed to reach it. Our camera technology is based on an novel imaging technique called Primal-Dual Coding pioneered by the PIs, and enables numerous applications in 3D scanning, visual recognition, biometrics, inspection and scientific imaging. We have already demonstrated the potential of the underlying principle of operation using off-the-shelf components. With this project we seek to unlock our camera technology's the true potential by (1) fabricating custom CMOS imagers, (2) designing and building new mobile camera platforms, and (3) developing the associated hardware and software architectures to support them.