Quantization Enabled Privacy Protection in Decentralized Stochastic Optimization
Quantization Enabled Privacy Protection in Decentralized Stochastic Optimization
复制标题
DOI：
10.1109/tac.2022.3198030
复制
发表时间：
2022-08
影响因子：
6.8
通讯作者：
Yongqiang Wang;T. Başar
中科院分区：
文献类型：
--
作者：
Yongqiang Wang;T. Başar
通过使多个代理在没有中央协调器的情况下合作解决全局优化问题，分散随机优化在机器学习、控制和传感器网络等不同领域越来越受到关注。由于相关数据通常包含敏感信息，如用户位置和个人身份，隐私保护已成为实现分散随机优化的关键需求。在本文中，我们提出了一种分散的随机优化算法，即使在存在与量化输入幅度成比例的严重量化误差的情况下，也能够保证可证明的收敛精度。结果适用于凸和非凸目标函数，并使我们能够利用积极的量化方案来混淆共享信息，因此，在不失去可证明的优化准确性的情况下实现隐私保护。事实上，通过将任意值量化为三个数值水平的随机三元量化方案，我们实现了分散随机优化中基于量化的严格差分隐私，这是以前没有报道过的。结合所提出的量化方案，该算法首次保证了分散随机优化中严格的差分隐私性，同时又不损失可证明的收敛精度。分布式估计问题的仿真结果以及在基准机器学习数据集上进行分散学习的数值实验证实了该方法的有效性。
By enabling multiple agents to cooperatively solve a global optimization problem in the absence of a central coordinator, decentralized stochastic optimization is gaining increasing attention in areas as diverse as machine learning, control, and sensor networks. Since the associated data usually contain sensitive information, such as user locations and personal identities, privacy protection has emerged as a crucial need in the implementation of decentralized stochastic optimization. In this article, we propose a decentralized stochastic optimization algorithm that is able to guarantee provable convergence accuracy even in the presence of aggressive quantization errors that are proportional to the amplitude of quantization inputs. The result applies to both convex and nonconvex objective functions, and enables us to exploit aggressive quantization schemes to obfuscate shared information and, hence, enables privacy protection without losing provable optimization accuracy. In fact, by using a stochastic ternary quantization scheme, which quantizes any value to three numerical levels, we achieve quantization-based rigorous differential privacy in decentralized stochastic optimization, which has not been reported before. In combination with the presented quantization scheme, the proposed algorithm ensures, for the first time, rigorous differential privacy in decentralized stochastic optimization without losing provable convergence accuracy. Simulation results for a distributed estimation problem as well as numerical experiments for decentralized learning on a benchmark machine learning dataset confirm the effectiveness of the proposed approach.