RI: AF: Small: Collaborative Research: Differentially Private Learning: From Theory to Applications

实用的隐私保护机器学习方法目前在医疗、金融和消费者应用等方面至关重要。该项目的目的是开发实用的私有机器学习算法，这些算法可以由任何持有敏感数据的领域的从业者轻松实现，同时保持强大的隐私保证。这项研究将扩展现有的差分隐私的严格理论保证，以达到现代机器学习算法在具体实际环境中的要求。因此，所产生的智力价值跨越了从理论到实际算法的所有方式。由此产生的方法有可能使许多高影响领域的现有现实世界的应用程序受益。PI与医疗从业者和研究人员进行了几次持续和成功的合作，并将在高影响力的医疗应用中对真实的患者数据进行评估。所有算法都将作为开源软件公开提供。这两个PI都致力于积极雇用少数民族和让本科生参与研究。差分隐私（DP）现在被认为是统计隐私中最严格和最有可能使用的概念之一，并已成为一个成熟的研究领域。 这项研究的目的是为实用的机器学习算法提供可靠的隐私保证，这对于保护自愿提供敏感数据用于研究目的的个人来说是非常宝贵的。我们确定了几个具有高影响潜力的领域，并提出了四个具体的研究重点。(1)私人因果推理。因果推理是机器学习中最有前途的新方向之一，最近已经变得实用。然而，一些最有趣的因果问题涉及医疗或政府政策数据，这些数据本质上是敏感的。我们建议联合最近在这两个领域的突破（因果推理和DP），并得出一个实际的和理论上合理的方法，以确保差异私人因果推理。(2)贝叶斯全局优化的隐私。深度学习的成功为贝叶斯全局优化（BGO）的超参数调整带来了激增的人气。同时，最近的出版物已经将差分隐私的稳定性与自适应数据分析中的泛化联系起来。我们建议将这些最新的发展结合起来，并使用DP的见解来改进BGO的泛化。在这里，我们不是保护个人隐私泄露，而是保护算法不被过度拟合--允许在“隐私”和功效之间进行权衡。(3)私有通信高效分布式学习。为了应对分布在多台机器上的数据的增长，我们的目标是为监督和无监督学习问题设计实用的私有和通信高效的算法。这项工作将建立在我们最近的分布式学习和聚类算法的工作。(4)实用的私人主动学习。在大数据时代，机器学习及其应用领域都对设计最有效地利用可用数据的主动学习算法产生了极大的兴趣，同时最大限度地减少了对人工干预的需求。最近，在理解统计和计算原理（包括PI的工作）方面取得了令人兴奋的成果。这项研究将开发新的基础和新的实用的有根据的主动学习算法，不仅在统计和计算上有效，而且还具有差异化的隐私。

Practical privacy-preserving machine learning methods are currently of critical importance in medical, financial and consumer applications, among others. The aim of this project is to develop practical private machine learning algorithms that can be easily implemented by practitioners in any field that holds sensitive data, while keeping robust privacy guarantees. The proposed research will extend the existing rigorous theoretical guarantees of differential privacy to reach the requirements of modern machine learning algorithms in concrete practical settings. The generated intellectual merit therefore spans all the way from the theory to practical algorithms. The resulting methods have the potential to benefit existing real-world applications in many high impact domains. The PIs have several ongoing and successful collaborations with medical practitioners and researchers and will evaluate the resulting algorithms on real patient data in high impact medical applications. All algorithms will be made publicly available as open source. Both PIs are dedicated towards actively hiring minorities and involving undergraduate students in research.Differential privacy (DP) is now recognized as one of the most rigorous and potentially usable notions of statistical privacy, and has become a full-fledged research field. The aim of this research is to provide reliable privacy guarantees for practical machine learning algorithms, which is invaluable to protect individuals who volunteer their sensitive data for research purposes. We identify several areas with high impact potential and propose four concrete research thrusts. (1) Private Causal Inference. Causal inference is one of the most promising new directions in machine learning, that recently has become practical. Some of the most interesting causal questions deal however with medical or government policy data, which are inherently sensitive. We propose to unite the recent breakthroughs in both fields (causal inference and DP) and derive a practical and theoretically sound method to ensure differentially private causal inference. (2) Privacy for Bayesian Global Optimization. The success of deep learning has created a surge in popularity for Bayesian Global Optimization (BGO) for hyper-parameter tuning. Simultaneously, recent publications have tied the stability properties of differential privacy to generalization in adaptive data analysis. We propose to unite these recent developments and improve the generalization of BGO using insights from DP. Here, we are not protecting individuals from privacy leaks, but algorithms from overfitting-allowing for fine trade-offs of "privacy" vs. efficacy. (3) Private Communication-Efficient Distributed Learning. In response to the growth of data distributed over multiple machines, we aim to design practical private and communication-efficient algorithms for supervised and unsupervised learning problems. This work will build off our recent work on distributed learning and clustering algorithms. (4) Practical Private Active Learning. In the age of big data, there has been tremendous interest both in machine learning and its application areas on designing active learning algorithms that most efficiently utilize the available data, while minimizing the need for human intervention. Recently there have been exciting results on understanding statistical and computational principles (including work by the PIs). This research will develop new foundations and new practical well-founded active learning algorithms that are not only statistically and computationally efficient, but also differentially private.