Limit Distribution Theory for KL divergence and Applications to Auditing Differential Privacy
Limit Distribution Theory for KL divergence and Applications to Auditing Differential Privacy
复制标题
DOI：
10.1109/isit54713.2023.10206925
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Sreejith Sreekumar;Ziv Goldfeld;Kengo Kato
中科院分区：
文献类型：
--
作者：
Sreejith Sreekumar;Ziv Goldfeld;Kengo Kato
Kullback-Leibler（KL）散度是概率分布之间的差异度量，在信息论、统计学和机器学习中起着核心作用。虽然有许多方法可以从数据中估计这个量，但量化估计误差波动的极限分布理论在很大程度上是模糊的。在本文中，我们关闭这个差距，确定充分条件的人口分布的分布极限的存在和特征的限制变量。这些结果被用来推导高斯平滑KL发散的单样本和双样本极限定理，无论是在零和替代。最后，极限分布结果的审计差分隐私的应用，提出和分析的显着性水平和权力对当地的替代品。
The Kullback-Leibler (KL) divergence is a discrepancy measure between probability distribution that plays a central role in information theory, statistics and machine learning. While there are numerous methods for estimating this quantity from data, a limit distribution theory which quantifies fluctuations of the estimation error is largely obscure. In this paper, we close this gap by identifying sufficient conditions on the population distributions for the existence of distributional limits and characterizing the limiting variables. These results are used to derive one- and two-sample limit theorems for Gaussian-smoothed KL divergence, both under the null and the alternative. Finally, an application of the limit distribution result to auditing differential privacy is proposed and analyzed for significance level and power against local alternatives.