RI: Medium: End-to-end Computational Sensing

动物有很多不同的方式。在无脊椎动物和小型爬行动物中尤其如此，进化产生了特别小，专门和高效的视觉系统。模仿这种专门的效率需要工程师所谓的“计算传感器”，这意味着传感器的光学和算法协同工作，以提高精度，同时降低传感器的尺寸和功耗要求。该项目将探索新一代视觉传感器的创建，这些传感器使用超表面作为其光学元件。超表面是一种半透明的薄片，上面有纳米级的结构图案，这些结构被定制成以特定的方式与光相互作用。除了比传统光学器件更小、更轻之外，它们还允许对光的各种特性进行前所未有的控制，包括相位、光谱和偏振。该项目将创建数学工具和软件工具，使工程师能够使用元表面创建新的视觉传感器。它还将利用这些工具来创建小型视觉传感器的原型，模仿跳蛛的深度感知能力，这些传感器非常小而高效。该项目将增加传感器设计过程中可以考虑的光学元件的种类。它还将使新的光学系统能够自动学习，而不必手动设计。这两者都将遵循光学和计算的统一考虑，该项目将通过将元表面视为前馈网络的通常计算层之前的光学处理层来实现。该项目将创建数学工具，允许元表面形状通过反向传播进行优化，同时优化捕获后计算的参数和超参数。在系统层面，该项目将生产被动快照深度传感器和快照RGBD传感器的原型，这些传感器非常小且节能。这些传感器将使用薄的毫米直径超表面和最小量的捕获后计算。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Animals see in many different ways. This is especially true in invertebrates and small reptiles, where evolution has produced vision systems that are particularly small, specialized, and efficient. Mimicking this specialized efficiency requires what engineers call "computational sensors," meaning sensors whose optics and algorithms work synergistically to improve accuracy while reducing sensor size and power requirements. This project will explore the creation of a new generation of visual sensors that use metasurfaces for their optics. Metasurfaces are ultrathin translucent sheets patterned with nanoscale structures that are tailored to interact with light in specific ways. In addition to being smaller and lighter than conventional optics, they allow unprecedented control of light's various properties, including phase, spectrum, and polarization. The project will create mathematical tools and software tools that allow engineers to create new visual sensors using metasurfaces. It will also use these tools to create prototypes of small visual sensors that mimick the depth-sensing capabilities of jumping spiders, which are remarkably small and efficient.The project will increase the variety of optics that can be considered during sensor design. It will also enable new optics to be learned automatically instead of having to be manually designed. Both of these will follow from the unified consideration of optics and computation, which this project will accomplish by treating metasurfaces as optical processing layers that precede the usual computational layers of a feed-forward network. The project will create mathematical tools that allow metasurface shapes to be optimized by back-propagation, in conjunction with optimizing the parameters and hyper-parameters of post-capture computations. At the systems level, the project will produce prototypes of passive snapshot depth sensors and snapshot RGBD sensors that are very small and power efficient. These sensors will use thin, millimeter-diameter metasurfaces and minimal amounts of post-capture computation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.