Vector Computing with Tensor Cores and Custom Accelerators in FPGA Overlays

点击翻译按钮获取中文摘要

With all of the IoT devices and sensors, from weather monitors to accelerometers to video cameras, the world is experiencing explosive data growth. Unless the data is immediately processed into a useful enduring form, the raw logs quickly lose their value. For example, the continuous data generated by video cameras becomes nearly useless if it must be searched as raw pixels after a recording is made. Instead, it is important to process data as it is generated, recording useful metadata along with the raw data. For example, tasks like object detection, classification, and speech recognition produce useful, searchable content making future retrieval possible. To scale such systems, it is best to perform these compute-intensive tasks as close to the sensor as possible, before it reaches the data center. This research is about the design of area-efficient, possibly portable, compute accelerators and processors that can be used in edge devices and end devices. In particular, it is important to accelerate processing of raw data produced by sensors such as audio and image/video, both at the sample-level (eg, pixel processing), up to understanding interpreting and creating actionable information using techniques in audio filtering and recognition, computer vision, and artificial intelligence. Accelerating frameworks already popular with users, such as OpenCV, TensorFlow and PyTorch, is most important. In pursuing the design of such accelerators and their programming systems, this research leverage the PI's experience in vector processors, FPGA overlays, neural network accelerators, and CAD tools. Past research has produced 4 generations of different vector processor architectures, where each generation was fully functional and implemented in FPGAs. The latest generation, VectorBlox MXP, was the foundation for a startup company that was purchased by a world-leading vendor of processors and FPGAs. The PI has also been an active participant in the development of the RISC-V processor extensions for vector processing and cache management. This research will continue the work by developing a complete processor and compute accelerator ecosystems, starting from the RISC-V specification and lessons learned with MXP. This accelerator will be vector-based, but generalized to support more parallelism using tensor cores, custom compute units, and better support for MIMD-style and pipeline parallelism. Leveraging the familiar tools of Python, LLVM and Linux, the research create new programming support systems, languages, and compilers to generate code for the new accelerators produced as FPGA overlays. Some parts of the hardware system will be auto-generated from techniques such as high-level synthesis, while others will be manually built. The end result will be easy-to-program systems that achieve high performance through effective use of parallelism.