Software Stack for General-Purpose Tensor Processing Units

神经网络在许多应用领域的广泛应用为开发更快的张量代数运算的专门架构解决方案创造了市场条件。这些操作是深度神经网络训练和推理的核心。该行业已经提供了各种各样的建筑解决方案。这种架构解决方案被称为张量处理单元（tpu）、神经网络处理器、深度学习加速器、人工智能芯片，通常基于称为收缩阵列的多处理单元拓扑，并具有加速多维阵列计算的共同目标。一个关键的理想特性是增加从内存带入处理单元的数据的重用。这项创新研究计划的前提是，为tpu开发的架构结构也可以用于通用数值计算应用。与GPU设计的演变类似，tpu的广泛使用将导致这些加速器的演变。这种进化反过来也将使它们更适合神经网络本身。虽然针对tpu存在许多编程解决方案（语言和库）和编译器，但用于神经处理的软件堆栈的开发仍处于起步阶段。最近，广泛使用的LLVM编译基础架构的创建者Chris Lattner提出了多层次中间表示（Multi-Level Intermediate Representation， MLIR），它有可能在编译过程的更高层次上实现更高效和更有效的优化。b谷歌有一个研发团队，致力于开发其采用所需的MLIR基础设施。然而，所有最初的努力都集中在TensorFlow的替代解决方案上，TensorFlow是谷歌目前提供的软件堆栈。这项提议的研究通过从传统的高性能计算（HPC）编程语言（如C或Fortran）创建编译路径来扩展tpu的软件堆栈，潜在地增加了OpenMP指令，以便在tpu中执行。主要目标：1；开发程序分析，以发现数值计算中的程序段（循环巢），这些程序段可以从tpu中可用的体系结构块的执行中受益。2. 创建可用于估计在TPU中执行计算时的性能和功耗的盈利能力分析。这些盈利能力分析还将用于确定哪些代码转换可以应用于代码，以提高其在TPU中的性能。3. 开发编译时代码转换，使给定的循环更适合在TPU硬件中执行。4. 扩展现有的ir，以更好地改善数值计算到tpu的映射。

The widespread adoption of neural networks in many application areas has created the market conditions for the development of specialized architectural solutions for the faster computation of tensor algebraic operations. These operations are at the core of both training and inference for deep neural networks. The industry has responded with a diverse offering of architectural solutions. Known as Tensor Processing Units (TPUs), Neural Network Processors, Deep-Learning Accelerators, Artificial Intelligence Chips, such architectural solutions are often based on a multi-processing-unit topology called systolic arrays and have the common goal of accelerating multi-dimensional array computations. A key desirable feature is to increase the reuse of data that has been brought from memory into the processing units. The premise of this innovative research program is that the architectural constructs developed for TPUs can also be deployed for general-purpose numerical computing applications. Similarly to what happened in the evolution of GPU design, the broader utilization of TPUs will lead to the evolution of these accelerators. This evolution will, in turn, make them also better-suited for neural networks themselves. While many programming solutions -- languages and libraries -- and compilers exist for TPUs, the development of a software stack for neural processing is still incipient. Recently Chris Lattner, the creator fo the widely used LLVM compilation infrastructure, has proposed the Multi-Level Intermediate Representation (MLIR) that has the potential of enabling more efficient and more effective optimizations at a higher level in the compilation process. Google has an R&D team working on the development of the MLIR infrastructure that is needed for its adoption. However, all the initial effort is focused on a replacement solution for TensorFlow, the current software stack offered by Google. The proposed research expands the software stack for TPUs by creating compilation paths from traditional High-Performance Computing (HPC) programming languages, such as C or Fortran, potentially augmented with OpenMP directives, for execution in TPUs. Main goals: 1. Develop program analysis to discover program segments (loop nests) in numerical computing that can benefit from execution in the architectural blocks available in TPUs. 2. Create profitability analysis that can be used to estimate the performance, and power consumption, when a computation is executed in a TPU. These profitability analysis will also be used to determine which code transformations could be applied to the code to improve its performance in a TPU. 3. Develop compile-time code transformations that can make a given loop next more suitable for execution in TPU hardware. 4. Extend existing IRs to better improve the mapping of numerical computations to TPUs.