An end-to-end Differentially Private Latent Dirichlet Allocation Using a Spectral Algorithm
An end-to-end Differentially Private Latent Dirichlet Allocation Using a Spectral Algorithm
复制标题
DOI：
--
复制
发表时间：
2018-05
期刊：
影响因子：
--
通讯作者：
Seyed-Alireza Esmaeili;Furong Huang
中科院分区：
文献类型：
--
作者：
Seyed-Alireza Esmaeili;Furong Huang
我们提供了一个端到端的差分私有频谱算法学习LDA，基于矩阵/张量分解，并建立理论保证估计模型参数的效用/一致性。频谱算法由多个算法步骤组成，称为“{边缘}”，可以向其中注入噪声以获得差分隐私。我们确定\n {子集的边缘}，命名为“{配置}"，这样，在这样的子集中的所有边缘添加噪声保证差分隐私的端到端频谱算法。我们的特点是相对于输入的边缘的灵敏度，从而估计的噪声量被添加到每个边缘的任何所需的隐私级别。然后，我们的特点，为每个配置的效用损失作为注入噪声的函数。总的来说，通过结合灵敏度和效用特性，我们获得了LDA的端到端差分私有频谱算法，并确定了在任何特定制度下优于其他人的相应配置。我们是第一个在LDA学习所需的差分隐私水平下实现效用保证的公司。总的来说，我们的方法系统地优于差分私人变分推理。
We provide an end-to-end differentially private spectral algorithm for learning LDA, based on matrix/tensor decompositions, and establish theoretical guarantees on utility/consistency of the estimated model parameters. The spectral algorithm consists of multiple algorithmic steps, named as "{edges}", to which noise could be injected to obtain differential privacy. We identify \emph{subsets of edges}, named as "{configurations}", such that adding noise to all edges in such a subset guarantees differential privacy of the end-to-end spectral algorithm. We characterize the sensitivity of the edges with respect to the input and thus estimate the amount of noise to be added to each edge for any required privacy level. We then characterize the utility loss for each configuration as a function of injected noise. Overall, by combining the sensitivity and utility characterization, we obtain an end-to-end differentially private spectral algorithm for LDA and identify the corresponding configuration that outperforms others in any specific regime. We are the first to achieve utility guarantees under the required level of differential privacy for learning in LDA. Overall our method systematically outperforms differentially private variational inference.