SPX: Secure, Highly-Parallel Training of Deep Neural Networks in the Cloud Using General-Purpose Shared-Memory Platforms

社会开始见证深度神经网络(DNN)的使用爆炸式增长，对人类生活的许多方面产生了重大影响，包括健康、财务、家庭生活和娱乐。为了培训DNN，从业者更喜欢使用GPU，最近还使用了专门的硬件加速器。尽管构成了数据中心的主体？S在计算资源方面，通用共享内存多处理器一直被认为是没有吸引力的平台。在这个项目中，首席调查员(PI)认为这些平台具有很高的潜力。因此，这个项目将开发新的技术来显著提高共享内存多处理器在训练DNN中的性能。目前，共享内存服务器之所以引人注目，有几个原因：它们可以支持高度的并行性，它们是通用的，易于编程，并提供灵活的、细粒度的核心间通信。然而，有效地使用共享内存服务器来训练DNN带来了巨大的挑战。首先，细粒度同步仍然昂贵，延迟也不是微不足道的。此外，当DNN培训迁移到云中多个用户共享同一物理共享内存平台的环境时，隐私和完整性成为主要关注的问题。为了克服这些挑战，该项目将协同解决架构和安全问题。在架构方面，它将增加一个高度并行的共享内存服务器，支持同步、数据移动、数据共享和DNN稀疏结构化。在安全方面，它将调查共享内存服务器如何造成新的隐私和完整性威胁(例如，泄露DNN？S稀疏结构和强制错误的模型生成)，以及如何防御这些威胁。这个项目是什么？S更广泛的影响是帮助实现？为每个人提供神经网络培训。通过使无处不在且易于编程的平台成为运行这些重要的新兴工作负载的可行且安全的目标。

Society is beginning to witness an explosion in the use of Deep Neural Networks (DNNs), with major impacts on many facets of human life, including health, finances, family life, and entertainment. To train DNNs, practitioners have preferred to use GPUs and, recently, specialized hardware accelerators.  Despite constituting the bulk of a data center?s compute resources, general-purpose shared-memory multiprocessors have been regarded as unattractive platforms. In this project, the Principal Investigators (PIs) think that these platforms have high potential. Consequently, this project will develop new techniques to dramatically improve shared-memory multiprocessor performance in training DNNs.  Already, shared-memory servers are compelling for several reasons: they can support a high-degree of parallelism, are general-purpose and easy to program, and provide flexible, fine-grain inter-core communication.  However, efficiently using shared-memory servers to train DNNs imposes significant challenges. First, fine-grain synchronization is still expensive, and latencies are non-trivial. In addition, when DNN training moves to an environment with multiple users sharing the same physical shared-memory platform in the cloud, privacy and integrity become major concerns.To overcome these challenges, this project will synergistically address architecture and security issues.  On the architecture side, it will augment a highly-parallel shared-memory server with support for synchronization, data movement, data sharing, and DNN sparsity structuring.  On the security side, it will investigate how shared-memory servers create novel privacy and integrity threats (for example, leaking the DNN?s sparse structure and forcing incorrect model generation), and how to defend against those threats.  The project?s broader impact is to help enable ?neural network training for everyone,? by making a ubiquitous and easy-to-program platform a viable and safe target for running these important, emerging workloads.