SPX: Secure, Highly-Parallel Training of Deep Neural Networks in the Cloud Using General-Purpose Shared-Memory Platforms

社会开始见证深度神经网络（DNN）的爆炸式使用，对人类生活的许多方面产生重大影响，包括健康，财务，家庭生活和娱乐。为了训练DNN，从业者更喜欢使用GPU，最近还使用了专门的硬件加速器。 尽管构成了数据中心的大部分？的计算资源，通用共享内存多处理器已被视为没有吸引力的平台。在该项目中，主要研究者（PI）认为这些平台具有很高的潜力。因此，该项目将开发新技术，以显着提高训练DNN的共享内存多处理器性能。 共享内存服务器之所以引人注目，有几个原因：它们可以支持高度并行性，通用且易于编程，并提供灵活的细粒度内核间通信。 然而，有效地使用共享内存服务器来训练DNN带来了重大挑战。首先，细粒度同步仍然是昂贵的，而延迟是不平凡的。此外，当DNN培训转移到多个用户共享云中同一物理共享内存平台的环境时，隐私和完整性成为主要问题。为了克服这些挑战，该项目将协同解决架构和安全问题。 在架构方面，它将通过支持同步、数据移动、数据共享和DNN稀疏结构来增强高度并行的共享内存服务器。 在安全方面，它将调查共享内存服务器如何创建新的隐私和完整性威胁（例如，泄露DNN？的稀疏结构和强制不正确的模型生成），以及如何防御这些威胁。 项目？更广泛的影响是帮助实现？每个人的神经网络训练通过使一个无处不在且易于编程的平台成为运行这些重要的新兴工作负载的可行且安全的目标。

Society is beginning to witness an explosion in the use of Deep Neural Networks (DNNs), with major impacts on many facets of human life, including health, finances, family life, and entertainment. To train DNNs, practitioners have preferred to use GPUs and, recently, specialized hardware accelerators.  Despite constituting the bulk of a data center?s compute resources, general-purpose shared-memory multiprocessors have been regarded as unattractive platforms. In this project, the Principal Investigators (PIs) think that these platforms have high potential. Consequently, this project will develop new techniques to dramatically improve shared-memory multiprocessor performance in training DNNs.  Already, shared-memory servers are compelling for several reasons: they can support a high-degree of parallelism, are general-purpose and easy to program, and provide flexible, fine-grain inter-core communication.  However, efficiently using shared-memory servers to train DNNs imposes significant challenges. First, fine-grain synchronization is still expensive, and latencies are non-trivial. In addition, when DNN training moves to an environment with multiple users sharing the same physical shared-memory platform in the cloud, privacy and integrity become major concerns.To overcome these challenges, this project will synergistically address architecture and security issues.  On the architecture side, it will augment a highly-parallel shared-memory server with support for synchronization, data movement, data sharing, and DNN sparsity structuring.  On the security side, it will investigate how shared-memory servers create novel privacy and integrity threats (for example, leaking the DNN?s sparse structure and forcing incorrect model generation), and how to defend against those threats.  The project?s broader impact is to help enable ?neural network training for everyone,? by making a ubiquitous and easy-to-program platform a viable and safe target for running these important, emerging workloads.