Collaborative Research: CNS Core: Medium: Cross-Layer Design of Video Analytics for the Internet of Things

物联网（IoT）的出现催生了许多新的应用，从增强现实和自动驾驶汽车，到监控和无收银员零售商店。这些应用程序不断地从物联网设备（如传感器、摄像头和雷达）收集视频流。他们的目标是通过运行复杂的视频分析任务，如计算人数和识别视频流中的车牌，来理解视频内容，从而做出明智的决策。这些视频分析任务通常运行一系列计算资源，包括物联网设备、设备附近的边缘集群和远程云，通过具有动态带宽和延迟的网络连接。该项目将实现一个高性能视频分析框架，可以实时、高精度、大规模地支持各种物联网应用。该项目的关键思想是通过跨应用程序、计算和网络的联合优化，为物联网设备实现视频分析。目前的解决方案通常侧重于分离的优化，这会导致对分析查询的回答不准确，计算资源的使用效率低下，并且当网络条件发生变化时性能会下降。本项目的视频分析框架将(1)利用网络层信息和物理信息来调整视频分析中的参数，以优化任务准确性，而不是网络带宽、延迟或体验质量；(2)为分析任务分配计算资源，通过分布式时间跟踪和运行时调度来满足多维任务级服务级目标。(3)考虑网络和计算约束，重新设计视频分析和编码算法。该项目将在系统之上构建和测试具有代表性的视频分析应用程序，以展示其能力。该项目将促进机器学习研究社区和系统/网络研究社区之间的互动，并为视频分析提供新颖的算法和高效的网络系统。该项目还将吸引代表性不足的群体和本科生参与研究。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The emergence of the Internet of Things (IoT) enables many new applications ranging from augmented reality and self-driving cars, to surveillance and cashier-less retail stores. These applications continuously collect video streams from IoT devices, such as sensors, cameras, and radars. They aim to understand the video content to make intelligent decisions, by running sophisticated video analytics tasks, such as counting people and recognizing license plates in the video streams. These video analytics tasks often run a collection of computing resources including IoT devices, edge clusters near the devices and the remote cloud, connected through networks with dynamic bandwidth and latency. This project will enable a high-performance video analytics framework that can support a variety of IoT applications in real-time, with high accuracy, and at scale. The key idea of this project is to enable video analytics for IoT devices by joint optimizations across application, computing, and networking. Today’s solutions often focus on separated optimization, which leads to inaccurate answers to analytical queries, inefficient use of computing resources, and performance degrades when network condition changes. This project's video analytics framework will (1) leverage both network layer information and physical information to tune the parameters in video analytics, in order to optimize task accuracy, instead of network bandwidth, latency or quality of experience, (2) allocate computing resources for analytics tasks to meet multi-dimensional task-level service-level objectives with distributed time tracking and runtime scheduling, and (3) redesign video analytics and encoding algorithms by considering the network and computing constraints. This project will build and test representative video analytics applications on top of the system to demonstrate its capability. The project will facilitate the interactions between the machine learning research community and the systems/networking research community, and result in novel algorithms and efficient networked systems for video analytics. The project will also engage underrepresented groups and undergraduates in research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.