CNS Core: Small: UbiVision: Ubiquitous Machine Vision with Adaptive Wireless Networking and Edge Computing

无线宽带和人工智能（AI）等技术的渗透正在推动家庭、工业和商业领域迅速采用网络摄像机。这些摄像机，如监控摄像机、仪表盘摄像机和可穿戴摄像机，可以捕获大量的视觉数据，这些数据可以转化为有价值的信息，用于公共安全、自动驾驶、服务机器人、增强/混合现实、辅助生活等。以及在保护用户隐私的同时共享来自无处不在的摄像头的有用信息。该项目使用来自无线网络，计算机视觉和边缘计算的技术和观点来分析和解决无处不在的摄像头系统中的问题，促进跨学科研究，为本科生和研究生提供独特的培训计划，并具有很高的潜力引入变革性技术，使新的现实生活中的产品和服务。该项目旨在实现无处不在的机器视觉（UbiVision），并有效利用网络摄像机进行信息提取和共享。为此，本文研究了三个基础性的研究问题：1）如何动态地管理多个技术领域中高度耦合的资源和功能：摄像机功能、网络资源和边缘服务器上的计算资源; 2）如何为资源受限的智能摄像机设计自适应和高效的机器视觉算法;以及3）如何设计可靠的机器学习框架，以在边缘服务器上进行强大的视觉分析。首先，结合传统优化和新兴机器学习技术的优点，设计了一种新的无模型端到端资源编排方法，以提高无线网络和计算的效率。其次，开发了一种新的通用卷积神经网络（CNN）和相应的CNN优化方法，用于智能相机上的高效多任务特征学习。第三，创新师生网络学习范式，开发出内存和计算效率高的机器视觉算法，能够在各种不利条件下实现强大的性能，这些不利条件是由变化的网络条件和有限的服务器计算预算造成的。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Penetration of technologies such as wireless broadband and artificial intelligence (AI) is propelling a rapid adoption of network cameras across the household, industrial, and commercial sectors. These cameras such as surveillance cameras, dash cameras, and wearable cameras can capture voluminous amounts of visual data that can be turned into valuable information for public safety, autonomous driving, service robots, augmented/mixed reality, assisted living, etc. To reach the potential, new methods are needed for efficiently and effectively extracting, transferring, and sharing useful information from ubiquitous cameras while preserving user privacy. This project uses techniques and perspectives from wireless networking, computer vision, and edge computing to analyze and solve the problems in ubiquitous camera systems, fosters interdisciplinary research, provides a unique training program for undergraduate and graduate students, and has a high potential to introduce transformative technologies that enable new real-life products and services. This project aims to realize ubiquitous machine vision (UbiVision) and enable efficient utilization of networked cameras for information extraction and sharing. Toward this end, three fundamental research problems are investigated: 1) how to dynamically manage highly coupled resources and functions across multiple technology domains: camera functions, network resources, and computation resources on edge servers; 2) how to design adaptive and efficient machine vision algorithms for resource-constrained smart cameras; and 3) how to engineer reliable machine learning frameworks for robust vision analysis on edge servers. First, a new model-free end-to-end resource orchestration method is designed to improve the efficiency of wireless networking and computing by combining the merits of conventional optimization and emerging machine learning techniques. Second, a novel universal convolution neural network (CNN) and corresponding CNN optimization methods are developed for efficient multi-task feature learning on smart cameras. Third, a teacher-student network learning paradigm is innovated to develop memory and computation efficient machine vision algorithms that are able to achieve robust performance under various adverse conditions caused by varying network conditions and limited server computation budgets.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.