Differentially Private SGDA for Minimax Problems
Differentially Private SGDA for Minimax Problems
复制标题
DOI：
--
复制
发表时间：
2022-01
期刊：
影响因子：
--
通讯作者：
Zhenhuan Yang;Shu Hu;Yunwen Lei;Kush R. Varshney;Siwei Lyu;Yiming Ying
中科院分区：
文献类型：
--
作者：
Zhenhuan Yang;Shu Hu;Yunwen Lei;Kush R. Varshney;Siwei Lyu;Yiming Ying
随机梯度下降上升法（SGDA）及其变种一直是解决极大极小问题的主力。然而，在对比研究的随机梯度下降（SGD）与差分隐私（DP）的约束，有很少的工作，了解推广（效用）的SGDA与DP约束。在本文中，我们使用的算法稳定性的方法来建立DP-SGDA在不同的设置的泛化（效用）。特别是，对于凹凸设置，我们证明了DP-SGDA可以实现最优效用率的弱原始-对偶人口风险光滑和非光滑的情况下。据我们所知，这是DP-SGDA在非光滑情况下的第一个已知结果。我们进一步提供了其效用分析的非凸强凹设置，这是有史以来第一次已知的结果，在原始人口的风险。这种非凸设置的收敛性和推广的结果是新的，即使在非私人设置。最后，数值实验证明了DP-SGDA的有效性，凸和非凸的情况下。
Stochastic gradient descent ascent (SGDA) and its variants have been the workhorse for solving minimax problems. However, in contrast to the well-studied stochastic gradient descent (SGD) with differential privacy (DP) constraints, there is little work on understanding the generalization (utility) of SGDA with DP constraints. In this paper, we use the algorithmic stability approach to establish the generalization (utility) of DP-SGDA in different settings. In particular, for the convex-concave setting, we prove that the DP-SGDA can achieve an optimal utility rate in terms of the weak primal-dual population risk in both smooth and non-smooth cases. To our best knowledge, this is the first-ever-known result for DP-SGDA in the non-smooth case. We further provide its utility analysis in the nonconvex-strongly-concave setting which is the first-ever-known result in terms of the primal population risk. The convergence and generalization results for this nonconvex setting are new even in the non-private setting. Finally, numerical experiments are conducted to demonstrate the effectiveness of DP-SGDA for both convex and nonconvex cases.