CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU
CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU
复制标题
DOI：
10.1109/sp40001.2021.00098
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Sijun Tan;Brian Knott;Yuan Tian;David J. Wu
中科院分区：
文献类型：
--
作者：
Sijun Tan;Brian Knott;Yuan Tian;David J. Wu
我们介绍了CryptGPU，这是一个用于隐私保护的机器学习系统，它在GPU(图形处理单元)上实现所有操作。正如图形处理器在现代深度学习的成功中发挥了关键作用一样，它们对于实现可扩展的隐私保护深度学习也是必不可少的。在这项工作中，我们首先引入一个新的接口，将秘密共享值(在离散域中)上的密码运算无损地嵌入到浮点运算中，这些运算可以由高度优化的线性代数CUDA内核处理。然后，我们确定了一系列对GPU友好的密码协议，以实现对GPU上的线性和非线性运算的隐私保护评估。我们的微基准测试表明，我们基于GPU的专用卷积协议比基于CPU的类似协议快150倍以上；对于REU激活功能等非线性操作，我们基于GPU的协议比其CPU模拟协议快约10倍。有了CryptGPU，我们支持对超过6000万个参数的卷积神经网络进行私人推理和训练，并处理像ImageNet这样的大型数据集。与已有协议相比，我们的协议在大型网络和数据集的私有推理方面获得了2×8倍的改进。对于私人培训，我们实现了比以前最先进的水平提高6到36倍。我们的工作不仅展示了完全在GPU上执行安全多方计算(MPC)以实现快速隐私保护机器学习的可行性，而且强调了设计能够充分利用GPU计算能力的新MPC原语的重要性。
We introduce CryptGPU, a system for privacy-preserving machine learning that implements all operations on the GPU (graphics processing unit). Just as GPUs played a pivotal role in the success of modern deep learning, they are also essential for realizing scalable privacy-preserving deep learning. In this work, we start by introducing a new interface to losslessly embed cryptographic operations over secret-shared values (in a discrete domain) into floating-point operations that can be processed by highly-optimized CUDA kernels for linear algebra. We then identify a sequence of "GPU-friendly" cryptographic protocols to enable privacy-preserving evaluation of both linear and non-linear operations on the GPU. Our microbenchmarks indicate that our private GPU-based convolution protocol is over 150× faster than the analogous CPU-based protocol; for non-linear operations like the ReLU activation function, our GPU-based protocol is around 10× faster than its CPU analog. With CryptGPU, we support private inference and training on convolutional neural networks with over 60 million parameters as well as handle large datasets like ImageNet. Compared to the previous state-of-the-art, our protocols achieve a 2× to 8× improvement in private inference for large networks and datasets. For private training, we achieve a 6× to 36× improvement over prior state-of-the-art. Our work not only showcases the viability of performing secure multiparty computation (MPC) entirely on the GPU to newly enable fast privacy-preserving machine learning, but also highlights the importance of designing new MPC primitives that can take full advantage of the GPU’s computing capabilities.