RI: Medium: End-to-end Computational Sensing

动物有许多不同的视觉方式。在无脊椎动物和小型爬行动物中尤其如此，它们的进化产生了特别小、专门和高效的视觉系统。模仿这种特殊的效率需要工程师们所说的“计算传感器”，这意味着传感器的光学和算法协同工作，以提高精度，同时减少传感器的尺寸和功率要求。该项目将探索新一代视觉传感器的创建，这些传感器使用超表面作为其光学元件。超表面是一种超薄的半透明薄片，上面有纳米级结构的图案，可以以特定的方式与光相互作用。除了比传统光学器件更小、更轻之外，它们还可以前所未有地控制光的各种特性，包括相位、光谱和偏振。该项目将创建数学工具和软件工具，使工程师能够使用元表面创建新的视觉传感器。它还将利用这些工具制造小型视觉传感器的原型，模仿跳蜘蛛的深度感知能力，跳蜘蛛非常小而高效。该项目将增加传感器设计过程中可考虑的光学器件的多样性。它还将使新的光学系统能够自动学习，而不必手动设计。这两者都将遵循光学和计算的统一考虑，该项目将通过将超表面作为前馈网络中通常计算层之前的光学处理层来实现。该项目将创建数学工具，允许通过反向传播优化超表面形状，同时优化捕获后计算的参数和超参数。在系统层面，该项目将生产非常小且节能的被动快照深度传感器和快照RGBD传感器原型。这些传感器将使用薄的、毫米直径的超表面和最少量的捕获后计算。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Animals see in many different ways. This is especially true in invertebrates and small reptiles, where evolution has produced vision systems that are particularly small, specialized, and efficient. Mimicking this specialized efficiency requires what engineers call "computational sensors," meaning sensors whose optics and algorithms work synergistically to improve accuracy while reducing sensor size and power requirements. This project will explore the creation of a new generation of visual sensors that use metasurfaces for their optics. Metasurfaces are ultrathin translucent sheets patterned with nanoscale structures that are tailored to interact with light in specific ways. In addition to being smaller and lighter than conventional optics, they allow unprecedented control of light's various properties, including phase, spectrum, and polarization. The project will create mathematical tools and software tools that allow engineers to create new visual sensors using metasurfaces. It will also use these tools to create prototypes of small visual sensors that mimick the depth-sensing capabilities of jumping spiders, which are remarkably small and efficient.The project will increase the variety of optics that can be considered during sensor design. It will also enable new optics to be learned automatically instead of having to be manually designed. Both of these will follow from the unified consideration of optics and computation, which this project will accomplish by treating metasurfaces as optical processing layers that precede the usual computational layers of a feed-forward network. The project will create mathematical tools that allow metasurface shapes to be optimized by back-propagation, in conjunction with optimizing the parameters and hyper-parameters of post-capture computations. At the systems level, the project will produce prototypes of passive snapshot depth sensors and snapshot RGBD sensors that are very small and power efficient. These sensors will use thin, millimeter-diameter metasurfaces and minimal amounts of post-capture computation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.