RI: AF: Small: Collaborative Research: Differentially Private Learning: From Theory To Applications

目前，实用的隐私保护机器学习方法在医疗、金融和消费者等应用中具有至关重要的作用。该项目的目标是开发实用的私人机器学习算法，这些算法可以由持有敏感数据的任何领域的从业者轻松实现，同时保持强大的隐私保障。提出的研究将扩展现有严格的差别隐私理论保证，以达到具体实际环境中现代机器学习算法的要求。因此，所产生的智力价值从理论到实际算法都是如此。由此产生的方法有可能使许多高影响领域的现有现实世界应用受益。PIS与医生和研究人员进行了几次持续且成功的合作，并将在高影响的医疗应用程序中评估产生的算法对真实患者数据的影响。所有算法都将以开源的形式公开提供。这两个PI都致力于积极雇用少数族裔和让本科生参与研究。差异隐私(DP)现在被认为是统计隐私中最严格和最有潜力使用的概念之一，并已成为一个成熟的研究领域。本研究的目的是为实用的机器学习算法提供可靠的隐私保证，这对于保护自愿将其敏感数据用于研究目的的个人来说是非常宝贵的。我们确定了几个具有高影响潜力的领域，并提出了四个具体的研究方向。(1)私人因果推论。因果推理是机器学习中最有前途的新方向之一，最近已经变得实用。然而，一些最有趣的因果问题涉及医疗或政府政策数据，这些数据本质上是敏感的。我们建议将这两个领域(因果推理和DP)的最新突破结合起来，并推导出一种实用的、理论上可靠的方法来确保差分私人因果推理。(2)贝叶斯全局优化的隐私性。深度学习的成功为贝叶斯全局优化(BGO)的超参数调整创造了一种流行的方式。同时，最近的出版物已经将差异隐私的稳定性属性与自适应数据分析中的泛化联系在一起。我们建议结合这些最新的发展，并利用DP的见解来改进BGO的泛化。在这里，我们保护的不是个人隐私的泄露，而是算法的过度匹配--允许在“隐私”和效率之间进行良好的权衡。(3)私密沟通--高效的分布式学习。为了应对分布在多台机器上的数据的增长，我们的目标是设计实用的私有和通信高效的算法来解决监督和非监督学习问题。这项工作将建立在我们最近在分布式学习和集群算法方面的工作的基础上。(4)实践性私人主动学习。在大数据时代，人们对机器学习及其应用领域一直非常感兴趣，目的是设计最有效地利用现有数据的主动学习算法，同时最大限度地减少对人工干预的需求。最近，在理解统计和计算原理方面取得了令人振奋的成果(包括私人投资机构的工作)。这项研究将开发新的基础和新的实用的良好基础的主动学习算法，不仅在统计和计算上有效，而且差分私有。

Practical privacy-preserving machine learning methods are currently of critical importance in medical, financial and consumer applications, among others. The aim of this project is to develop practical private machine learning algorithms that can be easily implemented by practitioners in any field that holds sensitive data, while keeping robust privacy guarantees. The proposed research will extend the existing rigorous theoretical guarantees of differential privacy to reach the requirements of modern machine learning algorithms in concrete practical settings. The generated intellectual merit therefore spans all the way from the theory to practical algorithms. The resulting methods have the potential to benefit existing real-world applications in many high impact domains. The PIs have several ongoing and successful collaborations with medical practitioners and researchers and will evaluate the resulting algorithms on real patient data in high impact medical applications. All algorithms will be made publicly available as open source. Both PIs are dedicated towards actively hiring minorities and involving undergraduate students in research.Differential privacy (DP) is now recognized as one of the most rigorous and potentially usable notions of statistical privacy, and has become a full-fledged research field. The aim of this research is to provide reliable privacy guarantees for practical machine learning algorithms, which is invaluable to protect individuals who volunteer their sensitive data for research purposes. We identify several areas with high impact potential and propose four concrete research thrusts. (1) Private Causal Inference. Causal inference is one of the most promising new directions in machine learning, that recently has become practical. Some of the most interesting causal questions deal however with medical or government policy data, which are inherently sensitive. We propose to unite the recent breakthroughs in both fields (causal inference and DP) and derive a practical and theoretically sound method to ensure differentially private causal inference. (2) Privacy for Bayesian Global Optimization. The success of deep learning has created a surge in popularity for Bayesian Global Optimization (BGO) for hyper-parameter tuning. Simultaneously, recent publications have tied the stability properties of differential privacy to generalization in adaptive data analysis. We propose to unite these recent developments and improve the generalization of BGO using insights from DP. Here, we are not protecting individuals from privacy leaks, but algorithms from overfitting-allowing for fine trade-offs of "privacy" vs. efficacy. (3) Private Communication-Efficient Distributed Learning. In response to the growth of data distributed over multiple machines, we aim to design practical private and communication-efficient algorithms for supervised and unsupervised learning problems. This work will build off our recent work on distributed learning and clustering algorithms. (4) Practical Private Active Learning. In the age of big data, there has been tremendous interest both in machine learning and its application areas on designing active learning algorithms that most efficiently utilize the available data, while minimizing the need for human intervention. Recently there have been exciting results on understanding statistical and computational principles (including work by the PIs). This research will develop new foundations and new practical well-founded active learning algorithms that are not only statistically and computationally efficient, but also differentially private.