CAREER: Fast Foveation: Bringing Active Vision into the Camera

注视点的流行，以及它在生活世界中的广泛多样性，清楚地表明这是一种有效的视觉设计策略。这个项目是关于复制注视，通过建造快速的相机，可以光学地将传感资源集中在周围世界感兴趣的区域。这样做可以提高计算机视觉智能系统的传感性能。在资源受限的平台上，如机器人或航天器，仅对感兴趣的领域进行自适应传感可以提高效率。该项目将创造能力，使各种传感应用。在整个项目期间，研究成果将整合到研究者的硬件/软件衔接课程中，重点放在摄像机校准等基本程序上。此外，还将启动一个名为LensLearning的项目，将注视点相机的概念推广到实验室之外。LensLearning包括通过佛罗里达大学的特殊项目来影响高中生。通过与佛罗里达大学（University of Florida）的相关项目合作，每年夏天还可以培训一名高中生和一名大四本科生，目标是在焦点相机研究中为未被充分代表的少数族裔提供机会。虽然人工注视点的想法已经通过缓慢的机械运动方式进行了探索，但在这个项目中，注视点相机和伴随的算法将会更快，因为它们利用了新一代的微机械光学，可以快速自适应地改变相机的分辨率。该项目的第一阶段包括建立快速注视点相机试验台，并通过考虑调制速度、相机视场、噪声、运动和远程效果的光学模型来表征动态场景快速注视点的基本限制。第二阶段涉及演示动态场景中快速注视点设置的各种跟踪优势，例如共定位系统和注视点相机阵列。模拟评估将使用广泛可用的数据集，通过比较处理能力和成像效率来完成。真实的评估也将在测试台上进行，从而发布一个新的日常物体动态场景的焦点数据集。在最后阶段，开发的系统和算法将通过结合大型基线和多模态系统来演示极端成像应用，展示诸如无眼镜眼球跟踪、黑暗环境成像和机器人快速面部成像等功能。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The prevalence of foveation, and the wide variety of it in the living world, makes it very clear that this is an effective visual design strategy. This project is about copying foveation, by building fast cameras that can optically concentrate sensing resources onto areas of interest in the world around them. Doing this can improve sensing performance for computer vision-enabled intelligent systems. On resource-constrained platforms, such as robots or spacecraft, adaptively sensing only on areas of interest improves efficiency. This project will create capability that enables a variety of sensing applications. Throughout the project timeline, research outcomes will be integrated in the investigator's hardware/software bridging courses, focused on fundamental procedures such as camera calibration. In addition, a program called LensLearning will be started, to spread foveating camera concepts beyond the lab. LensLearning includes impacting high-school students through special University of Florida programs with hands-on projects. It also enables the training of one high-school student and one undergraduate senior every summer through this project's timeline, by working with the University of Florida's associated programs, with the goal of giving opportunities to underrepresented minorities in foveated camera research.Although the idea of artificial foveation has been explored with slow, mechanical means of motion, in this project the foveating cameras and accompanying algorithms will be much faster because they exploit newly available, next generation micro-mechanical optics that can quickly and adaptively change the camera resolution. The first phase of this project involves building the fast foveating camera test-bed and characterizing the fundamental limits of fast foveation for dynamic scenes through an optical model that considers modulation speed, camera field-of-view, noise, motion and long-range effects. The second phase involves demonstrating tracking advantages in dynamic scenes with variants of the fast foveation setup, such as co-located systems and arrays of foveating cameras. Evaluations in simulation will be done using widely available datasets by comparing processing power and imaging efficiency. Real evaluation will also be done on the test bed, resulting in the release of a novel foveated dataset of dynamic scenes of everyday objects. In the last phase, the developed systems and algorithms will be used to demonstrate extreme imaging applications by combining both large baselines and co-located multimodal systems, showing capabilities such as glasses-free eye-tracking, imaging in dark environments and fast face imaging for robotics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.