Efficient Distributed DNN Training and Inference

近年来，深度学习在计算机视觉、机器翻译和语音识别等广泛的任务中取得了显著的成功，在机器学习社区内外引起了极大的关注。然而，由于高度的计算复杂性、大量的参数和迭代处理的大型数据集，训练这些模型需要几天到几周甚至几个月的时间才能完成。这种高计算成本使得分布式训练成为必要，以保证训练时间的合理性，同时也需要强大的硬件加速器，如通用gpu（图形处理单元）和tpu（张量处理单元）。在这个项目中，我们的目标是研究几种主要的方法来提高DNN训练和推理的性能和能源效率：(i)训练中的数据编码和压缩；（ii）推理中的数据编码和压缩；（iii）分布式深度神经网络训练的网络效率提高；（四）训练并行模型研究。我们还将探索使用华为NPU训练集群使用特定领域的ML加速的可能性。

In recent years, deep learning has attracted tremendous attention in the machine learning community and beyond by achieving notable success across a wide spectrum of tasks such as computer vision, machinetranslationand speech recognition. Training these models, however, take days to weeks or sometimes even months to finish because of the high degree of computational complexity, large number of parameters andlarge datasets iteratively processed. This high computation cost necessitates distributed training to keep the training time reasonable and also require powerful hardware accelerators such as General Purpose GPUs (graphics processing units) and TPUs (tensor processing units).In this project we aim to investigate several major ways to improve the performance and energy efficiency of DNN training and inference through: (i) data encoding and compression in training; (ii) data encoding and compression in inference; (iii) networking efficiency improvements in distributed DNN training; (iv) training parallelism model research. We will also going to explore the possibility of using domain-specific ML accleration using Huawei NPU training cluster.