Software Stack for General-Purpose Tensor Processing Units

通用张量处理单元的软件堆栈

• 批准号: RGPIN-2020-04006
• 负责人: Amaral, Jose
• 金额: $ 2.55万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=716765
• 关键词: Software; Stack; General; Purpose; Tensor

The widespread adoption of neural networks in many application areas has created the market conditions for the development of specialized architectural solutions for the faster computation of tensor algebraic operations. These operations are at the core of both training and inference for deep neural networks. The industry has responded with a diverse offering of architectural solutions. Known as Tensor Processing Units (TPUs), Neural Network Processors, Deep-Learning Accelerators, Artificial Intelligence Chips, such architectural solutions are often based on a multi-processing-unit topology called systolic arrays and have the common goal of accelerating multi-dimensional array computations. A key desirable feature is to increase the reuse of data that has been brought from memory into the processing units. The premise of this innovative research program is that the architectural constructs developed for TPUs can also be deployed for general-purpose numerical computing applications. Similarly to what happened in the evolution of GPU design, the broader utilization of TPUs will lead to the evolution of these accelerators. This evolution will, in turn, make them also better-suited for neural networks themselves. While many programming solutions -- languages and libraries -- and compilers exist for TPUs, the development of a software stack for neural processing is still incipient. Recently Chris Lattner, the creator fo the widely used LLVM compilation infrastructure, has proposed the Multi-Level Intermediate Representation (MLIR) that has the potential of enabling more efficient and more effective optimizations at a higher level in the compilation process. Google has an R&D team working on the development of the MLIR infrastructure that is needed for its adoption. However, all the initial effort is focused on a replacement solution for TensorFlow, the current software stack offered by Google. The proposed research expands the software stack for TPUs by creating compilation paths from traditional High-Performance Computing (HPC) programming languages, such as C or Fortran, potentially augmented with OpenMP directives, for execution in TPUs. Main goals: 1. Develop program analysis to discover program segments (loop nests) in numerical computing that can benefit from execution in the architectural blocks available in TPUs. 2. Create profitability analysis that can be used to estimate the performance, and power consumption, when a computation is executed in a TPU. These profitability analysis will also be used to determine which code transformations could be applied to the code to improve its performance in a TPU. 3. Develop compile-time code transformations that can make a given loop next more suitable for execution in TPU hardware. 4. Extend existing IRs to better improve the mapping of numerical computations to TPUs.

神经网络在许多应用领域的广泛采用为开发专门的架构解决方案创造了市场条件，以实现张量代数运算的更快计算。这些操作是深度神经网络训练和推理的核心。业界已经做出了回应，提供了多样化的架构解决方案。被称为张量处理单元（TPU），神经网络处理器，深度学习加速器，人工智能芯片，这种架构解决方案通常基于称为脉动阵列的多处理单元拓扑结构，并且具有加速多维阵列计算的共同目标。一个关键的期望特性是增加已经从存储器带入处理单元的数据的重用。 这项创新研究计划的前提是，为TPU开发的架构结构也可以部署为通用数值计算应用程序。与GPU设计的演变过程类似，TPU的更广泛利用将导致这些加速器的演变。反过来，这种进化也将使它们更适合神经网络本身。虽然TPU存在许多编程解决方案（语言和库）和编译器，但用于神经处理的软件栈的开发仍处于初期阶段。最近，广泛使用的LLVM编译基础设施的创建者Chris Lattner提出了多级中间表示（MLIR），该表示具有在编译过程中在更高级别上实现更高效和更有效优化的潜力。谷歌有一个研发团队致力于开发MLIR基础设施，这是采用MLIR所需要的。然而，所有最初的努力都集中在TensorFlow的替代解决方案上，TensorFlow是Google目前提供的软件堆栈。 拟议的研究通过从传统的高性能计算（HPC）编程语言（如C或Fortran）创建编译路径，扩展了TPU的软件堆栈，可能会增加OpenMP指令，用于在TPU中执行。 主要目标： 1.开发程序分析，以发现数值计算中的程序段（循环嵌套），这些程序段可以从TPU中可用的架构块中的执行中受益。 2.创建盈利能力分析，可用于评估在TPU中执行计算时的性能和功耗。这些盈利能力分析还将用于确定哪些代码转换可以应用于代码，以提高TPU中的性能。 3.开发编译时代码转换，使给定的循环更适合在TPU硬件中执行。 4.扩展现有IR，以更好地改进数值计算到TPU的映射。