CNS Core: Small: Software-Defined Video Analytics Pipeline: Enabling Resilient, High-Accuracy, and Resource-Effective Video Analytics

点击翻译按钮获取中文摘要

Significant progress in machine learning and computer vision techniques along with growth in Internet of Things, edge computing and high-bandwidth access networks such as 5G in recent years have led to the wide adoption of video analytics systems. Such systems deploy cameras in major cities in the US and around the world to support diverse applications in surveillance, transportation, public safety, health-care, retail, and home automation. A typical video analytics system deployment consists of a video analytics pipeline (VAP), where video cameras are deployed at different locations of interest such as airports and hospitals to continuously capture video streams and transport them over the network (e.g., 5G) to the cloud servers that perform video analytics processing. As the network condition, compute resource availability, and importantly the content of the captured video frames undergo changes over time, the VAP needs to be continuously adapted in order to support resilient, high-accuracy and resource-efficient video analytics applications. The large amount of proposed VAP adaptation design in recent years ignore the built-in frame/video processing configurability of modern cameras, rely on costly offline/online profiling, and are limited to simple frame/video adaptations such as frame rate tuning and down-sampling. This project aims to develop key technologies that enable a software-defined video analytics pipeline architecture that supports resilient, high-accuracy, resource-efficient video analytics using commodity reconfigurable network cameras widely available in the market today. It will develop (1) the first software-defined VAP abstraction that instills “intelligence” into the very first stage of a video analytics pipeline, the camera itself, (2) the first software architecture that enables fully automated, real-time adaptation of VAPs by exploiting reconfigurable cameras, which has the potential to significantly improve the resilience of video analytics systems to environmental condition changes around the camera, and (3) the first capability to jointly adapt complex camera parameters to optimize the accuracy and resource usage of multiple analytics tasks that share a VAP and hence its camera capture, which lowers the cost of VAP deployment.The proposed research will have direct, practical implications to the video analytics industry and large societal impact. (1) The proposed software-defined VAP architecture will provide a much needed reference system design and implementation of high-accuracy, resource-efficient VAPs that maximally exploit the in-built frame processing capabilities of modern network cameras, and thus has the potential to foster the proliferation and wide adoption of “smart” cameras in video analytics system deployment. (2) The technologies developed for enabling resilient, high-accuracy, resource-efficient and cost-efficient VAPs will foster wide adoption of many important societal VAP applications such as transportation, entertainment, health-care, retail, automotive, home automation, safety, and security. (3) Technically, this work will have a far-reaching impact beyond the area of optimizing video analytics systems by developing general software-defined architectures for optimizing other classes of remote sensing systems and applications based on smart sensors such as LiDARs and UWB sensors. The research team will actively disseminate and transfer the technologies developed to the video analytics industry, and help organize the annual IEEE Autonomous Unmanned Aerial Vehicles (UAV) Competition for high school students world-wide.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.