CNS Core: Medium: Characterization, Mitigation, and Management of Active 3D Camera Interference

CNS 核心：中：主动 3D 相机干扰的表征、缓解和管理

• 批准号: 2107060
• 负责人: Suman Banerjee
• 金额: $ 100万
• 依托单位: University of Wisconsin-Madison
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2107060&HistoricalAwards=false
• 关键词: CNS; Core; Medium; Characterization; Mitigation

This project focuses on interference encountered in active 3D camera networks. Active 3D cameras are gaining widespread use in many scenarios, primarily to understand location and presence of objects in vicinity of the camera. While expensive forms of it is common in autonomous vehicles for understanding the environment, technological improvements have led to these devices to be inexpensive, and suitable for integration in common mobile devices, such as smartphones and tablets. In such mobile devices, active 3D cameras enable many new ways that users can interact with the physical world. Active 3D cameras operate by sending light pulses in different directions, and by observing the reflections of these light pulses, can understand the properties of its environment. As more and more devices are equipped with active 3D cameras, chances of interference between multiple active 3D cameras go up. As the light pulses from different active 3D cameras "collide" in space and time, the inference about the environment is going to be inaccurate. In the case of autonomous vehicles this can lead to incorrect estimates of objects in the physical world leading to disastrous consequences. In the case of common mobile devices and their applications, such errors will lead to poor user experience and overall performance. While this problem has not been a significant issue thus far, due to limited number of active 3D cameras in our environment, the significant reduction in its cost will likely lead to an explosion of these devices. This project is designing methods by which such interference between active 3D cameras can be detected, mitigated, thereby preserving their usefulness in diverse scenarios. Beyond the core research activities, the project also involves significant educational components which include setup of a laboratory-oriented curriculum around 3D camera and wireless systems for students at both the undergraduate and graduate levels; creation of interactive, children-focused workshops on “fun with cameras” at public science events in the University Wisconsin-Madison campus; and engagement of undergraduates in research activities. The outreach plan involves engagement with industrial partners, public dissemination of results through project website, through open-sourced artifacts, and through conferences and journals, and in creating specific forums that encourage students from underrepresented groups towards STEM careers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

本项目的重点是有源3D摄像机网络中遇到的干扰。主动式3D摄像机在许多场景中得到了广泛的使用，主要用于了解摄像机附近物体的位置和存在。虽然在自动驾驶汽车中了解环境的昂贵形式很常见，但技术进步导致这些设备价格低廉，适合集成到智能手机和平板电脑等常见移动设备中。在这样的移动设备中，活动的3D摄像头使用户能够以许多新的方式与现实世界互动。主动式3D相机通过向不同方向发送光脉冲来运行，通过观察这些光脉冲的反射，可以了解其环境的属性。随着越来越多的设备配备主动式3D摄像头，多个主动式3D摄像头之间发生干扰的可能性也会增加。由于不同活动3D摄像机发出的光脉冲在空间和时间上“碰撞”，对环境的推断将是不准确的。在自动驾驶汽车的情况下，这可能会导致对物理世界中物体的错误估计，从而导致灾难性的后果。在常见的移动设备及其应用的情况下，这样的错误会导致用户体验和整体性能较差。虽然这个问题到目前为止还不是一个大问题，但由于我们环境中活动的3D摄像头数量有限，其成本的显著降低可能会导致这些设备的爆炸性增长。该项目正在设计方法，通过这些方法可以检测、减轻活动的3D摄像机之间的干扰，从而保持它们在不同场景中的实用性。除了核心研究活动外，该项目还涉及重要的教育内容，包括为本科生和研究生设置以实验室为导向的课程，围绕3D相机和无线系统，在威斯康星大学麦迪逊分校的公共科学活动中创建以儿童为重点的互动讲习班，讨论“玩相机的乐趣”，以及让本科生参与研究活动。外展计划涉及与行业合作伙伴的参与，通过项目网站、通过开源文物以及通过会议和期刊公开传播结果，并创建特定论坛，鼓励来自代表性不足群体的学生走向STEM职业。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

3D Scene Inference from Transient Histograms

• DOI: 10.48550/arxiv.2211.05094
• 发表时间: 2022-11
• 作者: Sacha Jungerman;A. Ingle;Yin Li;Mohit Gupta

When Two Cameras Are a Crowd

当两台相机拥挤时

• DOI: 10.1145/3588998
• 发表时间: 2023
• 期刊: Communications of the ACM
• 影响因子: 22.7
• 作者: Lee, Jongho;Gupta, Mohit;Krishnaswamy, Bhuvana;Banerjee, Suman
• 通讯作者: Banerjee, Suman

Unlocking the Performance of Proximity Sensors by Transient Histograms

通过瞬态直方图释放接近传感器的性能

• 发表时间: 2023
• 期刊: IEEE robotics automation letters
• 作者: Carter Sifferman;Yeping Wang;Mohit Gupta;Michael Gleicher
• 通讯作者: Michael Gleicher

Mitigating AC and DC Interference in Multi-ToF-Camera Environments

• DOI: 10.1109/tpami.2023.3307564
• 发表时间: 2023-08
• 期刊: IEEE Transactions on Pattern Analysis and Machine Intelligence
• 影响因子: 23.6
• 作者: Jongho Lee;Mohit Gupta

Compressive Single-Photon 3D Cameras

压缩式单光子 3D 相机

• DOI: 10.1109/cvpr52688.2022.01733
• 发表时间: 2022
• 期刊: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR
• 作者: Gutierrez-Barragan, Felipe;Ingle, Atul;Seets, Trevor;Gupta, Mohit;Velten, Andreas
• 通讯作者: Velten, Andreas