Collaborative Research: OAC Core: Advancing Low-Power Computer Vision at the Edge

这一方案使手机、无人机、物联网设备等低功耗边缘计算机能够造福社会。计算机视觉是一种自动分析图像和视频的技术。这些设备上的计算机视觉可以保证人类的安全，例如在工厂或建筑工地发现危险。该项目解决了阻碍在边缘设备上实际采用计算机视觉的两个挑战。第一个挑战是，目前的计算机视觉方法需要强大的计算机，但这些计算机距离太远，响应时间长。这个项目将计算机带到需要获取数据的地方。该项目使计算机视觉更高效，因此视觉数据可以通过手机和无人机等小型边缘设备进行分析。第二个挑战是为计算机视觉构建复杂的软件是困难的。该项目为新兴的计算机视觉技术提供软件工程支持。由于解决了这两个挑战，边缘计算机视觉可以变得可行。将计算机视觉（CV）引入网络边缘设备是实现NSF分布式网络基础设施目标的重要组成部分。该项目通过提高响应时间、减少网络覆盖需求和降低存储成本，使边缘CV成为可能，并实现科学和工程创新。该项目解决了阻碍基于边缘的CV向实践过渡的两个关键挑战。(1)该项目提高了CV的效率和边缘友好性。当前的CV技术（如深度神经网络）需要服务器级资源（如图形处理单元、千兆内存）；这些资源在边缘不可用。这个项目减少了CV所需的资源需求。该方法在处理视觉数据时考虑了可选的神经网络架构并消除了冗余。该项目还开发了特定于cv的分发技术，使边缘设备能够在大型视觉任务上进行协作。(2)本项目为CV技术提供软件工程支持。解决现实世界的CV问题需要设计新的CV应用程序，通常是通过在新设计中重新实现研究模型架构作为组件。该项目开发了一个用于低功耗平台的示例CV模型实现库。这些示例可以用作新的CV应用中的高质量组件。该项目确定了促进和抑制CV模型可重复性的因素。该项目还通过调查和采访低功耗CV专家以及研究他们的错误来确定工程最佳实践。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This proposal enables low-power edge computers, such as mobile phones, drones, and Internet-of-Things devices, to benefit society. Computer vision is the technology to automatically analyze images and videos. Computer vision on these devices can keep humans safe, for example by spotting dangers in a factory or at a construction site. This project addresses two challenges that hamper practical adoption of computer vision on edge devices. The first challenge is that current computer vision approaches require powerful computers, but these computers are too far away and have long response time. This project brings the computers to the places where data is acquired. The project makes computer vision more efficient, so that visual data can be analyzed by small edge devices like phones and drones. The second challenge is that building complex software for computer vision is difficult. This project provides software engineering support for emerging computer vision technologies. As a result of addressing these two challenges, computer vision on the edge can become feasible.Bringing computer vision (CV) to devices on the network edge is an essential component of realizing NSF's goal of distributed cyberinfrastructure. This project makes CV on the edge feasible and enables scientific and engineering innovation through improved response time, reduced need for network coverage, and decreased storage costs. This project solves two critical challenges that hinder the transition of edge-based CV into practice. (1) This project makes CV more efficient and edge-friendly. Current CV techniques (e.g., deep neural networks) assume server-class resources (such as graphics processing units, gigabytes of memory); these resources are not available at the edge. This project reduces the resource requirements needed for CV. The methods consider alternative neural network architectures and eliminate redundancies while processing visual data. This project also develops CV-specific distribution techniques to enable edge devices to collaborate on large vision tasks. (2) This project provides software engineering support for CV technologies. Solving real-world CV problems requires engineering new CV applications, often by re-implementing research model architectures as components in new designs. This project develops a library of exemplary CV model implementations for low-power platforms. These exemplars can be used as high-quality components in new CV applications. The project identifies factors that promote and inhibit the reproducibility of CV models. This project also identifies engineering best practices by surveying and interviewing experts in low-power CV and by studying their errors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.