CNS Core: Small: UbiVision: Ubiquitous Machine Vision with Adaptive Wireless Networking and Edge Computing

无线宽带和人工智能(AI)等技术的普及正在推动网络摄像头在家庭、工业和商业部门的快速采用。这些摄像头如监控摄像头、仪表盘摄像头和可穿戴摄像头可以捕获海量的视觉数据，这些数据可以转化为公共安全、自动驾驶、服务机器人、增强/混合现实、辅助生活等有价值的信息。为了发挥潜力，需要新的方法来高效和有效地从无处不在的摄像头中提取、传输和共享有用的信息，同时保护用户隐私。该项目使用无线网络、计算机视觉和边缘计算的技术和视角来分析和解决无处不在的摄像头系统中的问题，促进跨学科研究，为本科生和研究生提供独特的培训计划，并具有引入变革性技术的高潜力，使新的现实生活产品和服务成为可能。该项目旨在实现无处不在的机器视觉(UbiVision)，并使网络摄像机能够有效地用于信息提取和共享。为此，本文研究了三个基本问题：1)如何跨多个技术领域动态管理高度耦合的资源和功能：摄像机功能、网络资源和边缘服务器上的计算资源；2)如何为资源受限的智能摄像机设计自适应和高效的机器视觉算法；3)如何为边缘服务器上的鲁棒视觉分析设计可靠的机器学习框架。首先，结合传统优化技术和新兴的机器学习技术的优点，设计了一种新的无模型端到端资源编排方法，以提高无线网络和计算的效率。其次，针对智能摄像机的多任务特征学习问题，提出了一种新的通用卷积神经网络(CNN)及其优化方法。第三，创新了教师-学生网络学习范式，以开发内存和计算效率高的机器视觉算法，该算法能够在因网络条件变化和服务器计算预算有限而导致的各种不利条件下获得稳健的性能。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Penetration of technologies such as wireless broadband and artificial intelligence (AI) is propelling a rapid adoption of network cameras across the household, industrial, and commercial sectors. These cameras such as surveillance cameras, dash cameras, and wearable cameras can capture voluminous amounts of visual data that can be turned into valuable information for public safety, autonomous driving, service robots, augmented/mixed reality, assisted living, etc. To reach the potential, new methods are needed for efficiently and effectively extracting, transferring, and sharing useful information from ubiquitous cameras while preserving user privacy. This project uses techniques and perspectives from wireless networking, computer vision, and edge computing to analyze and solve the problems in ubiquitous camera systems, fosters interdisciplinary research, provides a unique training program for undergraduate and graduate students, and has a high potential to introduce transformative technologies that enable new real-life products and services. This project aims to realize ubiquitous machine vision (UbiVision) and enable efficient utilization of networked cameras for information extraction and sharing. Toward this end, three fundamental research problems are investigated: 1) how to dynamically manage highly coupled resources and functions across multiple technology domains: camera functions, network resources, and computation resources on edge servers; 2) how to design adaptive and efficient machine vision algorithms for resource-constrained smart cameras; and 3) how to engineer reliable machine learning frameworks for robust vision analysis on edge servers. First, a new model-free end-to-end resource orchestration method is designed to improve the efficiency of wireless networking and computing by combining the merits of conventional optimization and emerging machine learning techniques. Second, a novel universal convolution neural network (CNN) and corresponding CNN optimization methods are developed for efficient multi-task feature learning on smart cameras. Third, a teacher-student network learning paradigm is innovated to develop memory and computation efficient machine vision algorithms that are able to achieve robust performance under various adverse conditions caused by varying network conditions and limited server computation budgets.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.