CNS Core: Small: Not All Cameras are Created Equal: Systems Support for Highly Adaptive Video Analytics Pipelines

CNS 核心：小型：并非所有摄像机都是一样的：对高度自适应视频分析管道的系统支持

• 批准号: 2153449
• 负责人: Ravi Netravali
• 金额: $ 50万
• 依托单位: Princeton University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2153449&HistoricalAwards=false
• 关键词: CNS; Core; Small; Cameras; Created

The ubiquity of video camera deployments, coupled with steady improvements in computer vision algorithms, has given rise to a diverse range of video analytics applications. Use cases include surveillance, traffic scheduling, disaster response, and more. Yet despite their promise, video analytics deployments are far from widespread. A key reason is that video analysis is often prohibitively expensive: video is data-intensive, stressing the network, and analysis typically involves Deep Neural Networks (DNNs) to query video, requiring substantial compute resources. This project aims to design and implement practical video analytics systems that can adapt their execution to most efficiently utilize end-to-end compute and network resources, i.e., across cameras, servers, and the networks between them.The key insight underlying the proposed work is to adaptively place analytics tasks by leveraging frame-transforming techniques that are diverse in terms of resource requirements and accuracy, e.g., lightweight frame differencing versus expensive object detection DNNs. Along these lines, the project involves three synergistic directions. First, it rigorously classifies existing frame transforming techniques, investigating the correlation between their computation costs, potential data reduction, and impact on response accuracy. Second, it develops end-to-end systems that can automatically select the appropriate frame transforming technique to run on a camera with the goal of optimizing for response latency and accuracy given the available resources. Third, it develops techniques to extend adaptive video analytics to emerging camera settings, e.g., multi-camera, steerable, energy-harvesting; these systems rely on the extraction of spatial and temporal relationships between camera feeds to guide resource allocation decisions.The proposed research targets a large slice of the population (given the breadth of video analytics applications), and improves both the accessibility and potential of video analytics deployments. The developed systems enable affordable (but effective) video analytics for organizations of different scale, allowing them to make the most of their available resources. Furthermore, the work motivates novel applications that were previously deemed impractical, e.g., real-time monitoring of rural areas via energy-harvesting cameras. The project also involves outreach efforts to attract students from populations currently under-represented in computer science. Key to these efforts is magnifying the interdisciplinary nature of video analytics pipelines which span systems, networks, machine learning, and computer vision.The software and research artifacts designed as part of this project are released on a public website: http://web.cs.ucla.edu/~ravi/adaptive_video_analytics/. The site is regularly maintained and includes replication instructions and packages. Project data are kept on the site for at least 5 years after publication, with extensions based on public interest.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

摄像机部署的普遍性，加上计算机视觉算法的稳步改进，已经产生了各种各样的视频分析应用。用例包括监控、流量调度、灾难响应等。然而，尽管视频分析的前景看好，但其部署远未普及。一个关键原因是视频分析通常非常昂贵：视频是数据密集型的，给网络带来压力，分析通常涉及深度神经网络（DNN）来查询视频，需要大量的计算资源。该项目旨在设计和实现实用的视频分析系统，这些系统可以调整其执行以最有效地利用端到端计算和网络资源，即，所提出的工作的关键见解是通过利用在资源要求和准确性方面不同的帧转换技术来自适应地放置分析任务，例如，轻量级帧差分与昂贵的对象检测DNN。沿着这些思路，该项目涉及三个协同方向。首先，它严格分类现有的帧转换技术，调查它们的计算成本，潜在的数据减少，响应精度的影响之间的相关性。其次，它开发了端到端系统，可以自动选择适当的帧转换技术在相机上运行，目标是在给定可用资源的情况下优化响应延迟和准确性。第三，它开发了将自适应视频分析扩展到新兴相机设置的技术，例如，多摄像头、可操控、能量采集;这些系统依赖于提取摄像头馈送之间的空间和时间关系来指导资源分配决策。拟议的研究针对大部分人群（考虑到视频分析应用的广度），并提高视频分析部署的可访问性和潜力。开发的系统为不同规模的组织提供了经济实惠（但有效）的视频分析，使他们能够充分利用现有资源。此外，这项工作激发了以前被认为不切实际的新应用，例如，通过能源采集摄像头对农村地区进行实时监控。该项目还涉及外联工作，以吸引目前在计算机科学领域人数不足的学生。这些努力的关键是放大视频分析管道的跨学科性质，这些管道跨越系统，网络，机器学习和计算机视觉。作为该项目的一部分设计的软件和研究工件在公共网站上发布：http://web.cs.ucla.edu/~ravi/adaptive_video_analytics/。该网站定期维护，包括复制说明和软件包。项目数据将在网站上保存至少5年，并根据公众利益进行延期。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Boggart: Towards General-Purpose Acceleration of Retrospective Video Analytics

• 发表时间: 2021-06
• 作者: Neil Agarwal;R. Netravali

GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge

• 发表时间: 2022-01
• 作者: Arthi Padmanabhan;Neil Agarwal;Anand Iyer;Ganesh Ananthanarayanan;Yuanchao Shu;Nikolaos Karianakis;G. Xu;R. Netravali

Privid: Practical, Privacy-Preserving Video Analytics Queries

• 发表时间: 2021-06
• 作者: Frank Cangialosi;Neil Agarwal;V. Arun;Junchen Jiang;Srinivas Narayana;Anand D. Sarwate;Ravi Netravali

RECL: Responsive Resource-Efficient Continuous Learning for Video Analytics

RECL：视频分析的响应式资源高效持续学习

• 发表时间: 2023
• 期刊: 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23
• 作者: Khani, Mehrdad;Ananthanarayanan, Ganesh;Hsieh, Kevin;Jiang, Junchen;Netravali, Ravi;Shu, Yuanchao;Alizadeh, Mohammad;Bahl, Victor
• 通讯作者: Bahl, Victor

Understanding the potential of server-driven edge video analytics

• DOI: 10.1145/3508396.3512872
• 发表时间: 2022-03
• 期刊: Proceedings of the 23rd Annual International Workshop on Mobile Computing Systems and Applications
• 作者: Qizheng Zhang;Kuntai Du;Neil Agarwal;R. Netravali;Junchen Jiang