RI: Medium: End-to-end Computational Sensing

动物以许多不同的方式看东西。这在无脊椎动物和小型爬行动物中尤其如此，它们的进化产生了特别小、专门和有效的视觉系统。要模仿这种特殊的效率，需要工程师所说的“计算传感器”，这意味着传感器的光学和算法协同工作，以提高精度，同时减少传感器的大小和功率要求。这个项目将探索创造新一代使用超表面作为光学元件的视觉传感器。超表面是带有纳米级结构图案的超薄半透明薄片，这些结构是为与光线以特定方式相互作用而量身定做的。除了比传统光学器件更小、更轻之外，它们还允许前所未有地控制光的各种属性，包括相位、光谱和偏振。该项目将创建数学工具和软件工具，允许工程师使用元表面创建新的视觉传感器。它还将使用这些工具来创造小型视觉传感器的原型，这些传感器模仿跳跃蜘蛛的深度感知能力，这种能力非常小，而且效率很高。该项目将增加传感器设计中可以考虑的光学元件的种类。它还将使新的光学元件能够自动学习，而不是必须手动设计。这两者都将遵循光学和计算的统一考虑，该项目将通过将变形表面作为光学处理层来实现，该光学处理层先于前馈网络的通常计算层。该项目将创建数学工具，允许通过反向传播来优化变形表面形状，同时优化捕获后计算的参数和超参数。在系统层面，该项目将生产非常小且节能的无源快照深度传感器和快照RGBD传感器的原型。这些传感器将使用薄的、毫米直径的准表面和最少的捕获后计算。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Animals see in many different ways. This is especially true in invertebrates and small reptiles, where evolution has produced vision systems that are particularly small, specialized, and efficient. Mimicking this specialized efficiency requires what engineers call "computational sensors," meaning sensors whose optics and algorithms work synergistically to improve accuracy while reducing sensor size and power requirements. This project will explore the creation of a new generation of visual sensors that use metasurfaces for their optics. Metasurfaces are ultrathin translucent sheets patterned with nanoscale structures that are tailored to interact with light in specific ways. In addition to being smaller and lighter than conventional optics, they allow unprecedented control of light's various properties, including phase, spectrum, and polarization. The project will create mathematical tools and software tools that allow engineers to create new visual sensors using metasurfaces. It will also use these tools to create prototypes of small visual sensors that mimick the depth-sensing capabilities of jumping spiders, which are remarkably small and efficient.The project will increase the variety of optics that can be considered during sensor design. It will also enable new optics to be learned automatically instead of having to be manually designed. Both of these will follow from the unified consideration of optics and computation, which this project will accomplish by treating metasurfaces as optical processing layers that precede the usual computational layers of a feed-forward network. The project will create mathematical tools that allow metasurface shapes to be optimized by back-propagation, in conjunction with optimizing the parameters and hyper-parameters of post-capture computations. At the systems level, the project will produce prototypes of passive snapshot depth sensors and snapshot RGBD sensors that are very small and power efficient. These sensors will use thin, millimeter-diameter metasurfaces and minimal amounts of post-capture computation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.