AutoReP: Automatic ReLU Replacement for Fast Private Network Inference
AutoReP: Automatic ReLU Replacement for Fast Private Network Inference
复制标题
DOI：
10.1109/iccv51070.2023.00478
复制
发表时间：
2023-08
期刊：
影响因子：
--
通讯作者：
Hongwu Peng;Shaoyi Huang;Tong Zhou;Yukui Luo;Chenghong Wang;Zigeng Wang;Jiahui Zhao;Xiaowei Xie;Ang Li;Tony Geng;Kaleel Mahmood;Wujie Wen;Xiaolin Xu;Caiwen Ding
中科院分区：
文献类型：
--
作者：
Hongwu Peng;Shaoyi Huang;Tong Zhou;Yukui Luo;Chenghong Wang;Zigeng Wang;Jiahui Zhao;Xiaowei Xie;Ang Li;Tony Geng;Kaleel Mahmood;Wujie Wen;Xiaolin Xu;Caiwen Ding
使用加密隐私的机器学习 - 服务（MLAAS）的增长强调了客户的数据隐私和安全性问题（PI）。工作介绍了一种基于梯度的方法，可减少非线性操作员，并减轻这些问题的选择。 CIFAR-10），8.39％（74.92％，12.9k Relu预算，CIFAR-100）和9.45％（63.69％，55k Relu预算，小型预算，微型imimageNet）超过当前的最新方法，例如，snl的速度为76。 。
The growth of the Machine-Learning-As-A-Service (MLaaS) market has highlighted clients’ data privacy and security issues. Private inference (PI) techniques using cryptographic primitives offer a solution but often have high computation and communication costs, particularly with non-linear operators like ReLU. Many attempts to reduce ReLU operations exist, but they may need heuristic threshold selection or cause substantial accuracy loss. This work introduces AutoReP, a gradient-based approach to lessen non-linear operators and alleviate these issues. It automates the selection of ReLU and polynomial functions to speed up PI applications and introduces distribution-aware polynomial approximation (DaPa) to maintain model expressivity while accurately approximating ReLUs. Our experimental results demonstrate significant accuracy improvements of 6.12% (94.31%, 12.9K ReLU budget, CIFAR-10), 8.39% (74.92%, 12.9K ReLU budget, CIFAR-100), and 9.45% (63.69%, 55K ReLU budget, Tiny-ImageNet) over current state-of-the-art methods, e.g., SNL. Morever, AutoReP is applied to EfficientNet-B2 on ImageNet dataset, and achieved 75.55% accuracy with 176.1 × ReLU budget reduction. The codes are shared on Github1.