SHF: Small: A Unified Approach for Scheduling Computer Vision Dataflow Graphs

计算机视觉和机器学习的进步使自主系统和从图像和视频中提取知识的新能力得以持续发展。这些进步在很大程度上是由于一种为视觉和神经处理而设计的新型处理器所带来的计算能力的提高。这包括谷歌的视觉核心和张量处理器单元，高通的六角形DSP，NVIDIA的可编程视觉加速器和张量核心，以及英特尔的佳洁士处理器。在这些处理器中，与通用处理器和图形处理器相比，程序代码对底层处理和内存资源的控制要大得多。这大大减少了限制准确性能预测的不确定性，并为自动和便携性能优化的新方法打开了大门。尽管嵌入式视觉处理器的市场不断增长，但它们的相关编译器工具还处于萌芽阶段，依赖于使用试错法手动调整程序代码。该项目将建立在现有技术的基础上，为广泛类别的视觉处理器建立一个“通用”的前端。作为该项目的一部分，研究人员还将开发一个OpenVX1.2内核数据库(https://www.khronos.org/openvx/))和一个OpenVX基准测试套件。预计在开放源码中开发工具和基准将促进社区参与。该项目开发的人工产物将作为计算机网络边缘的物联网和机器学习新课程的教学基础设施。该项目正在开发一个新的程序编译流程，在该流程中，以平台无关的OpenVX表示编写的程序被转换为低级别表示，并显式地调度到底层硬件资源上。该转换由硬件行为的数学模型来管理，该硬件行为是从执行和检测一系列基准测试所生成的挖掘性能数据训练而来的。使用这种方法开发的编译器框架有可能取代目前使用的特别方法，并成为为视觉和神经处理器开发软件的强大工具，这将为下一代认知智能设备提供动力。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Advances in computer vision and machine learning are enabling continuous development of new capabilities in autonomous systems and knowledge extraction from images and video. These advances are possible in large part to increased computational capability brought about by a new class of processor designed for visual and neural processing. This includes Google's Visual Core and Tensor Processor Unit, Qualcomm's Hexagon DSP, Nvidia's Programmable Vision Accelerator and Tensor Core, and Intel's Crest processors. In these processors, the program code exerts far greater control of the underlying processing and memory resources than in general purpose and graphical processors. This serves to greatly reduce the uncertainties that would otherwise restrict accurate performance prediction and open the door for new approaches for automatic and portable performance optimization. Despite the growing market for embedded vision processors, their associated compiler tools are nascent, relying on hand-tuning program code using trial-and-error methods. This project will build upon existing technologies to build a "universal" front-end for a broad class of vision processors. As part of this project the researchers will also develop an database of OpenVX 1.2 kernels (https://www.khronos.org/openvx/) and an OpenVX benchmark suite. It is expected that developing both the tools and benchmarks in the open source will facilitate community involvement. The artifacts developed in this project will serve as a pedagogical infrastructure for a new course in Internet-of-Things and machine learning at the edge of the computer network.This project is developing a new program compilation flow, in which a program written in the platform-independent OpenVX representation is converted into a low-level representation and explicitly scheduled onto the underlying hardware resources. The conversion is governed by a mathematical model of the hardware behavior trained from mining performance data generated from executing and instrumenting a series of benchmarks. The compiler framework developed using this approach has the potential to replace the ad hoc methods currently in use and become a powerful tool for developing software for visual and neural processors, which will power a future generation of cognizant smart devices.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.