Vector Computing with Tensor Cores and Custom Accelerators in FPGA Overlays

随着所有的物联网设备和传感器，从天气监测器到加速计到摄像机，世界正在经历爆炸性的数据增长。除非立即将数据处理成有用的持久形式，否则原始日志很快就会失去价值。例如，摄像机产生的连续数据如果在记录后必须作为原始像素进行搜索，则变得几乎无用。相反，重要的是在生成数据时对其进行处理，将有用的元数据沿着记录在原始数据中。例如，像对象检测、分类和语音识别这样的任务会产生有用的、可搜索的内容，使未来的检索成为可能。要扩展此类系统，最好在传感器到达数据中心之前尽可能靠近传感器执行这些计算密集型任务。这项研究是关于设计面积高效，可能便携的计算加速器和处理器，可用于边缘设备和终端设备。特别是，重要的是要加速处理传感器产生的原始数据，如音频和图像/视频，无论是在样本级（如像素处理），直到理解解释和创建可操作的信息使用音频过滤和识别，计算机视觉和人工智能技术。加速已经受到用户欢迎的框架，如OpenCV，TensorFlow和PyTorch，是最重要的。在追求这样的加速器及其编程系统的设计，本研究利用PI的矢量处理器，FPGA覆盖，神经网络加速器和CAD工具的经验。过去的研究已经产生了4代不同的向量处理器架构，每一代都是功能齐全的，并在FPGA中实现。最新一代VectorBlox MXP是一家初创公司的基础，该公司被一家世界领先的处理器和FPGA供应商收购。PI还积极参与了用于矢量处理和缓存管理的RISC-V处理器扩展的开发。这项研究将通过开发一个完整的处理器和计算加速器生态系统来继续这项工作，从RISC-V规范和MXP的经验教训开始。这个加速器将基于向量，但一般化以支持更多的并行性，使用张量核心，自定义计算单元，以及更好地支持MIMD风格和流水线并行性。利用熟悉的Python、LLVM和Linux工具，该研究创建了新的编程支持系统、语言和编译器，为作为FPGA覆盖层产生的新加速器生成代码。硬件系统的某些部分将通过高级合成等技术自动生成，而其他部分将手动构建。最终的结果将是易于编程的系统，通过有效使用并行实现高性能。

With all of the IoT devices and sensors, from weather monitors to accelerometers to video cameras, the world is experiencing explosive data growth. Unless the data is immediately processed into a useful enduring form, the raw logs quickly lose their value. For example, the continuous data generated by video cameras becomes nearly useless if it must be searched as raw pixels after a recording is made. Instead, it is important to process data as it is generated, recording useful metadata along with the raw data. For example, tasks like object detection, classification, and speech recognition produce useful, searchable content making future retrieval possible. To scale such systems, it is best to perform these compute-intensive tasks as close to the sensor as possible, before it reaches the data center. This research is about the design of area-efficient, possibly portable, compute accelerators and processors that can be used in edge devices and end devices. In particular, it is important to accelerate processing of raw data produced by sensors such as audio and image/video, both at the sample-level (eg, pixel processing), up to understanding interpreting and creating actionable information using techniques in audio filtering and recognition, computer vision, and artificial intelligence. Accelerating frameworks already popular with users, such as OpenCV, TensorFlow and PyTorch, is most important. In pursuing the design of such accelerators and their programming systems, this research leverage the PI's experience in vector processors, FPGA overlays, neural network accelerators, and CAD tools. Past research has produced 4 generations of different vector processor architectures, where each generation was fully functional and implemented in FPGAs. The latest generation, VectorBlox MXP, was the foundation for a startup company that was purchased by a world-leading vendor of processors and FPGAs. The PI has also been an active participant in the development of the RISC-V processor extensions for vector processing and cache management. This research will continue the work by developing a complete processor and compute accelerator ecosystems, starting from the RISC-V specification and lessons learned with MXP. This accelerator will be vector-based, but generalized to support more parallelism using tensor cores, custom compute units, and better support for MIMD-style and pipeline parallelism. Leveraging the familiar tools of Python, LLVM and Linux, the research create new programming support systems, languages, and compilers to generate code for the new accelerators produced as FPGA overlays. Some parts of the hardware system will be auto-generated from techniques such as high-level synthesis, while others will be manually built. The end result will be easy-to-program systems that achieve high performance through effective use of parallelism.