Computational and Optical Processing Architectures for next-Generation Mobile Cameras

移动设备正在推动我们相互之间以及与周围世界互动的方式发生重大变化。它们正准备成为我们自己的延伸，感知物质世界的能力急剧增加。摄像头是移动设备的“眼睛”，其光学质量、速度和无线联网能力的不断提高催生了新的市场和新的用途(例如，基于图像的搜索和针对视障人士的辅助技术)。 尽管过去移动相机的工作主要集中在让它们“看得更清楚”(更精细的细节，每秒更多的图像，需要更少的光线)，但现在是时候问一问“移动相机还能看到什么”。这项拟议的工作将开发并实际演示使用一种前所未有的完全不同的相机可能的应用。我们的相机技术不仅将以远远超过现有解决方案的细节来拍摄三维世界(即它所看到的对象的距离和3D形状)，而且它还可以考虑光线到达它所遵循的3D路径。我们的相机技术基于PIS首创的名为Primal-Dual Coding的新型成像技术，可在3D扫描、视觉识别、生物识别、检测和科学成像中实现多种应用。我们已经展示了使用现成组件的基本操作原理的潜力。通过这个项目，我们寻求通过(1)制造定制的CMOS成像器，(2)设计和构建新的移动相机平台，以及(3)开发相关的硬件和软件架构来支持它们，从而释放我们相机技术的真正潜力。

Mobile devices are fuelling a major shift in how we interact with each other and with the world around us. They are poised to become an extension of ourselves, with drastically-increased capabilities for sensing the physical world. Cameras are the "eyes" of mobile devices, and continuous improvements in their optical quality, speed and wireless networking capabilities have spawned new markets and new uses (e.g., image-based search, and assistive technologies for the visually impaired). While past work in mobile cameras focused primarily on making them "see better" (finer detail, more images per second, requiring less light), it is now time to ask "what else can mobile cameras see". The proposed work will develop and practically demonstrate the applications possible with a radically-different camera that has never-seen-before abilities. Not only will our camera technology image the world in three dimensions (i.e., the distance and 3D shape of objects it "sees") at a detail that far surpasses existing solutions, but it can take into account the 3D path that light followed to reach it. Our camera technology is based on an novel imaging technique called Primal-Dual Coding pioneered by the PIs, and enables numerous applications in 3D scanning, visual recognition, biometrics, inspection and scientific imaging. We have already demonstrated the potential of the underlying principle of operation using off-the-shelf components. With this project we seek to unlock our camera technology's the true potential by (1) fabricating custom CMOS imagers, (2) designing and building new mobile camera platforms, and (3) developing the associated hardware and software architectures to support them.