LinGCN: Structural Linearized Graph Convolutional Network for Homomorphically Encrypted Inference
LinGCN: Structural Linearized Graph Convolutional Network for Homomorphically Encrypted Inference
复制标题
DOI：
10.48550/arxiv.2309.14331
复制
发表时间：
2023-09
期刊：
影响因子：
--
通讯作者：
Hongwu Peng;Ran Ran-Ran;Yukui Luo;Jiahui Zhao;Shaoyi Huang;Kiran Thorat;Tong Geng;Chenghong Wang;Xiaolin Xu;Wujie Wen;Caiwen Ding
中科院分区：
文献类型：
--
作者：
Hongwu Peng;Ran Ran-Ran;Yukui Luo;Jiahui Zhao;Shaoyi Huang;Kiran Thorat;Tong Geng;Chenghong Wang;Xiaolin Xu;Wujie Wen;Caiwen Ding
图形卷积网络（GCN）模型大小的增长已彻底改变了许多应用程序，超过了个人医疗保健和金融系统等领域的人类绩效。由于对客户数据的潜在对抗性攻击，GCN在云中的部署引起了隐私问题。为了解决安全问题，使用同态加密（HE）可以保护敏感的客户数据。但是，它在实际应用中引入了大量的计算开销。为了应对这些挑战，我们提出了lingcn，该框架旨在减少乘法深度并优化基于HE的GCN推断的性能。 LingCN围绕三个关键要素结构：（1）一种可区分的结构线性化算法，并由参数化的离散指标函数互补，并与模型权重共同训练以满足优化目标。该策略促进了细粒节点级非线性位置选择，从而导致具有最小化乘法深度的模型。 （2）通过二阶可训练的激活功能，紧凑的节点多项式替换策略，通过从基于全卢比的教师模型中的两级蒸馏方法转向上级融合。 （3）一种增强的HE解决方案，该解决方案可以使更细粒的操作员融合节点激活功能，从而进一步降低了基于HE的推断中的乘法水平消耗。我们在NTU-Xview骨骼关节数据集上进行的实验表明，LingCN在同派加密推理的潜伏期，准确性和可伸缩性方面表现出色，胜过诸如CryptoGCN之类的解决方案。值得注意的是，LingCN相对于CryptoGCN实现了14.2倍的延迟速度，同时保留了75％的推理精度，并且尤其是降低了乘法深度。
The growth of Graph Convolution Network (GCN) model sizes has revolutionized numerous applications, surpassing human performance in areas such as personal healthcare and financial systems. The deployment of GCNs in the cloud raises privacy concerns due to potential adversarial attacks on client data. To address security concerns, Privacy-Preserving Machine Learning (PPML) using Homomorphic Encryption (HE) secures sensitive client data. However, it introduces substantial computational overhead in practical applications. To tackle those challenges, we present LinGCN, a framework designed to reduce multiplication depth and optimize the performance of HE based GCN inference. LinGCN is structured around three key elements: (1) A differentiable structural linearization algorithm, complemented by a parameterized discrete indicator function, co-trained with model weights to meet the optimization goal. This strategy promotes fine-grained node-level non-linear location selection, resulting in a model with minimized multiplication depth. (2) A compact node-wise polynomial replacement policy with a second-order trainable activation function, steered towards superior convergence by a two-level distillation approach from an all-ReLU based teacher model. (3) an enhanced HE solution that enables finer-grained operator fusion for node-wise activation functions, further reducing multiplication level consumption in HE-based inference. Our experiments on the NTU-XVIEW skeleton joint dataset reveal that LinGCN excels in latency, accuracy, and scalability for homomorphically encrypted inference, outperforming solutions such as CryptoGCN. Remarkably, LinGCN achieves a 14.2x latency speedup relative to CryptoGCN, while preserving an inference accuracy of 75% and notably reducing multiplication depth.