CAREER: Fast Foveation: Bringing Active Vision into the Camera

职业：快速注视点：将主动视觉带入相机

• 批准号: 1942444
• 负责人: Sanjeev Koppal
• 金额: $ 51.93万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1942444&HistoricalAwards=false
• 关键词: CAREER; Fast; Foveation; Bringing; Active

The prevalence of foveation, and the wide variety of it in the living world, makes it very clear that this is an effective visual design strategy. This project is about copying foveation, by building fast cameras that can optically concentrate sensing resources onto areas of interest in the world around them. Doing this can improve sensing performance for computer vision-enabled intelligent systems. On resource-constrained platforms, such as robots or spacecraft, adaptively sensing only on areas of interest improves efficiency. This project will create capability that enables a variety of sensing applications. Throughout the project timeline, research outcomes will be integrated in the investigator's hardware/software bridging courses, focused on fundamental procedures such as camera calibration. In addition, a program called LensLearning will be started, to spread foveating camera concepts beyond the lab. LensLearning includes impacting high-school students through special University of Florida programs with hands-on projects. It also enables the training of one high-school student and one undergraduate senior every summer through this project's timeline, by working with the University of Florida's associated programs, with the goal of giving opportunities to underrepresented minorities in foveated camera research.Although the idea of artificial foveation has been explored with slow, mechanical means of motion, in this project the foveating cameras and accompanying algorithms will be much faster because they exploit newly available, next generation micro-mechanical optics that can quickly and adaptively change the camera resolution. The first phase of this project involves building the fast foveating camera test-bed and characterizing the fundamental limits of fast foveation for dynamic scenes through an optical model that considers modulation speed, camera field-of-view, noise, motion and long-range effects. The second phase involves demonstrating tracking advantages in dynamic scenes with variants of the fast foveation setup, such as co-located systems and arrays of foveating cameras. Evaluations in simulation will be done using widely available datasets by comparing processing power and imaging efficiency. Real evaluation will also be done on the test bed, resulting in the release of a novel foveated dataset of dynamic scenes of everyday objects. In the last phase, the developed systems and algorithms will be used to demonstrate extreme imaging applications by combining both large baselines and co-located multimodal systems, showing capabilities such as glasses-free eye-tracking, imaging in dark environments and fast face imaging for robotics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

视觉中心化的流行，以及生活世界中的各种各样，都清楚地表明这是一种有效的视觉设计策略。这个项目是关于复制foveation，通过建立快速相机，可以光学集中传感资源到他们周围的世界感兴趣的领域。这样做可以提高计算机视觉智能系统的传感性能。在资源受限的平台上，如机器人或航天器，仅在感兴趣的区域进行自适应感测可以提高效率。该项目将创建能够实现各种传感应用的能力。在整个项目时间轴中，研究成果将被整合到研究者的硬件/软件桥接课程中，重点是摄像机校准等基本程序。此外，还将启动一个名为LensLearning的项目，将视觉聚焦相机的概念传播到实验室之外。LensLearning包括通过佛罗里达大学的特殊项目和实践项目来影响高中生。它还通过与佛罗里达大学的相关项目合作，通过该项目的时间轴，每年夏天对一名高中生和一名本科生进行培训，目的是为在中心凹相机研究中代表性不足的少数民族提供机会。虽然人工中心凹的想法已经通过缓慢的机械运动方式进行了探索，在该项目中，视觉聚焦照相机和伴随的算法将快得多，因为它们利用了新的可用的下一代微机械光学器件，其可以快速且自适应地改变照相机分辨率。该项目的第一阶段包括建立快速视觉聚焦相机测试平台，并通过考虑调制速度、相机视场、噪声、运动和远程效应的光学模型来表征动态场景快速视觉聚焦的基本限制。第二阶段涉及展示在动态场景中使用快速视觉聚焦设置的变体（例如协同定位系统和视觉聚焦相机阵列）的跟踪优势。模拟评估将使用广泛可用的数据集，通过比较处理能力和成像效率来完成。真实的评估也将在测试台上进行，从而发布一个新颖的日常物体动态场景的中心凹数据集。在最后阶段，开发的系统和算法将用于通过结合大基线和协同定位多模式系统来展示极端成像应用，展示裸眼眼动追踪等功能，该奖项反映了NSF的法定使命，并被认为是值得通过评估使用基金会的智力价值和更广泛的支持。影响审查标准。

项目成果

Energy-Efficient Adaptive 3D Sensing

节能的自适应 3D 传感

• 发表时间: 2023
• 期刊: IEEE Conference on Computer Vision and Pattern Recognition
• 作者: Tilmon, Brevin;Sun, Zhanghao;Koppal, Sanjeev J.;Wu, Yicheng;Evangelidis, Georgios;Zahreddine, Ramzi;Krishnan, Gurunandan;Ma, Sizhuo;Wang, Jian
• 通讯作者: Wang, Jian

Fast Foveating Cameras for Dense Adaptive Resolution

快速注视点相机，实现密集自适应分辨率

• DOI: 10.1109/tpami.2021.3071588
• 发表时间: 2021
• 期刊: IEEE Transactions on Pattern Analysis and Machine Intelligence
• 影响因子: 23.6
• 作者: Tilmon, Brevin;Jain, Eakta;Ferrari, Silvia;Koppal, Sanjeev Jagannatha
• 通讯作者: Koppal, Sanjeev Jagannatha