Efficient Privacy-Preserving Stochastic Nonconvex Optimization
Efficient Privacy-Preserving Stochastic Nonconvex Optimization
复制标题
DOI：
--
复制
发表时间：
2019-10
期刊：
影响因子：
--
通讯作者：
Lingxiao Wang;Bargav Jayaraman;David Evans;Quanquan Gu
中科院分区：
文献类型：
--
作者：
Lingxiao Wang;Bargav Jayaraman;David Evans;Quanquan Gu
尽管已经开发了许多保护隐私的凸经验风险最小化 (ERM) 解决方案，但保护隐私的非凸 ERM 仍然是一个挑战。我们研究非凸 ERM，它采用最小化训练集上非凸损失函数的有限和的形式。我们提出了一种新的非凸 ERM 差分隐私随机梯度下降算法，该算法有效地实现了强大的隐私保证，并对其隐私和效用保证以及梯度复杂性进行了严格分析。我们的算法大大降低了梯度复杂性，同时匹配 Wang 等人给出的最佳先前效用保证。 （2017）。我们使用安全多方计算将我们的算法扩展到分布式设置，并表明分布式算法有可能在该设置中匹配集中式算法的隐私和实用性保证。我们对基准非凸 ERM 问题的实验表明，与之前使用相同隐私预算的差分隐私方法相比，在训练成本和效用收益方面均表现出优越的性能。
While many solutions for privacy-preserving convex empirical risk minimization (ERM) have been developed, privacy-preserving nonconvex ERM remains a challenge. We study nonconvex ERM, which takes the form of minimizing a finite-sum of nonconvex loss functions over a training set. We propose a new differentially private stochastic gradient descent algorithm for nonconvex ERM that achieves strong privacy guarantees efficiently, and provide a tight analysis of its privacy and utility guarantees, as well as its gradient complexity. Our algorithm substantially reduces gradient complexity while matching the best previous utility guarantee given by Wang et al. (2017). We extend our algorithm to the distributed setting using secure multi-party computation, and show it is possible for a distributed algorithm to match the privacy and utility guarantees of a centralized algorithm in this setting. Our experiments on benchmark nonconvex ERM problems demonstrate superior performance in terms of both training cost and utility gains compared with previous differentially private methods using the same privacy budgets.