Collaborative Research: OAC Core: Advancing Low-Power Computer Vision at the Edge

合作研究：OAC Core：推进边缘低功耗计算机视觉

• 批准号: 2107020
• 负责人: George Thiruvathukal
• 金额: $ 25万
• 依托单位: Loyola University of Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2107020&HistoricalAwards=false
• 关键词: Collaborative; Research; OAC; Core; Advancing

This proposal enables low-power edge computers, such as mobile phones, drones, and Internet-of-Things devices, to benefit society. Computer vision is the technology to automatically analyze images and videos. Computer vision on these devices can keep humans safe, for example by spotting dangers in a factory or at a construction site. This project addresses two challenges that hamper practical adoption of computer vision on edge devices. The first challenge is that current computer vision approaches require powerful computers, but these computers are too far away and have long response time. This project brings the computers to the places where data is acquired. The project makes computer vision more efficient, so that visual data can be analyzed by small edge devices like phones and drones. The second challenge is that building complex software for computer vision is difficult. This project provides software engineering support for emerging computer vision technologies. As a result of addressing these two challenges, computer vision on the edge can become feasible.Bringing computer vision (CV) to devices on the network edge is an essential component of realizing NSF's goal of distributed cyberinfrastructure. This project makes CV on the edge feasible and enables scientific and engineering innovation through improved response time, reduced need for network coverage, and decreased storage costs. This project solves two critical challenges that hinder the transition of edge-based CV into practice. (1) This project makes CV more efficient and edge-friendly. Current CV techniques (e.g., deep neural networks) assume server-class resources (such as graphics processing units, gigabytes of memory); these resources are not available at the edge. This project reduces the resource requirements needed for CV. The methods consider alternative neural network architectures and eliminate redundancies while processing visual data. This project also develops CV-specific distribution techniques to enable edge devices to collaborate on large vision tasks. (2) This project provides software engineering support for CV technologies. Solving real-world CV problems requires engineering new CV applications, often by re-implementing research model architectures as components in new designs. This project develops a library of exemplary CV model implementations for low-power platforms. These exemplars can be used as high-quality components in new CV applications. The project identifies factors that promote and inhibit the reproducibility of CV models. This project also identifies engineering best practices by surveying and interviewing experts in low-power CV and by studying their errors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这一提议使手机、无人机和物联网设备等低功耗边缘计算机能够造福社会。计算机视觉是自动分析图像和视频的技术。这些设备上的计算机视觉可以保护人类的安全，例如通过在工厂或建筑工地发现危险。该项目解决了阻碍计算机视觉在边缘设备上实际采用的两个挑战。第一个挑战是，目前的计算机视觉方法需要功能强大的计算机，但这些计算机距离太远，响应时间长。该项目将计算机带到获取数据的地方。该项目提高了计算机视觉的效率，因此可以通过手机和无人机等小型边缘设备来分析视觉数据。第二个挑战是，为计算机视觉构建复杂的软件是困难的。该项目为新兴的计算机视觉技术提供软件工程支持。由于解决了这两个挑战，边缘的计算机视觉成为可能。将计算机视觉(CV)带到网络边缘的设备是实现NSF分布式网络基础设施目标的重要组成部分。该项目通过缩短响应时间、减少对网络覆盖的需求以及降低存储成本，使边缘CV变得可行，并支持科学和工程创新。该项目解决了阻碍基于边缘的简历向实践过渡的两个关键挑战。(1)此项目使CV更加高效和边缘友好。当前的CV技术(例如，深度神经网络)假定服务器级的资源(例如图形处理单元、千兆字节的存储器)；这些资源在边缘不可用。该项目减少了简历所需的资源需求。该方法考虑了其他神经网络结构，并在处理视觉数据时消除了冗余。该项目还开发了特定于CV的分发技术，以使边缘设备能够在大型视觉任务中进行协作。(2)本项目为CV技术提供软件工程支持。解决现实世界的简历问题需要设计新的简历应用程序，通常是通过在新设计中重新实现研究模型架构作为组件来实现的。该项目为低功耗平台开发了一个模范CV模型实施库。这些样本可以用作新简历应用中的高质量组件。该项目确定了促进和抑制CV模型重现性的因素。该项目还通过调查和采访低功率简历中的专家并研究他们的错误来确定工程最佳实践。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Establishing Trust in Vehicle-to-Vehicle Coordination: A Sensor Fusion Approach

建立车辆间协调的信任：传感器融合方法

• DOI: 10.1109/di-cps56137.2022.00008
• 发表时间: 2022
• 期刊: 2022 2nd Workshop on Data-Driven and Intelligent Cyber-Physical Systems for Smart Cities Workshop (DI-CPS
• 作者: Veselsky, Jakob;West, Jack;Ahlgren, Isaac;Thiruvathukal, George K.;Klingensmith, Neil;Goel, Abhinav;Jiang, Wenxin;Davis, James C.;Lee, Kyuin;Kim, Younghyun
• 通讯作者: Kim, Younghyun

Are You Really Muted?: A Privacy Analysis of Mute Buttons in Video Conferencing Apps

• DOI: 10.48550/arxiv.2204.06128
• 发表时间: 2022-04
• 期刊: Proc. Priv. Enhancing Technol.
• 作者: Yucheng Yang;Jack West;G. Thiruvathukal;Neil Klingensmith;Kassem Fawaz

Snapshot Metrics Are Not Enough: Analyzing Software Repositories with Longitudinal Metrics

快照指标还不够：使用纵向指标分析软件存储库

• DOI: 10.1145/3551349.3559517
• 发表时间: 2022
• 期刊: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering
• 作者: Synovic, Nicholas M.;Hyatt, Matt;Sethi, Rohan;Thota, Sohini;Shilpika;Miller, Allan J.;Jiang, Wenxin;Amobi, Emmanuel S.;Pinderski, Austin;Läufer, Konstantin
• 通讯作者: Läufer, Konstantin