AF: Small: Advances in Private Optimization

现代人工智能（AI）系统通常是使用大型数据集构建的，这些数据集可能是由个人通过互联网贡献数据生成的，例如通过产品评级，评论或其他在线交互。因此，至关重要的是，这样的人工智能系统能够保护其数据被使用的个人的隐私：外部观察者不应该通过使用人工智能系统来了解任何个人的任何信息。该项目将研究隐私和性能之间权衡的基本限制，以构建尽可能高性能而不影响隐私的AI系统。该项目的成果不仅将提高那些已经从他们的数据中学习的产品中受益的人的安全性，而且还将通过使更敏感或更脆弱的个人安全地参与来减少人工智能中的偏见。在技术层面上，该项目将开发新的差分私有随机优化算法。近年来，人们对私有优化的兴趣激增，但私有非凸优化（训练神经网络所需的）理论却令人惊讶地不发达。对于这种设置，典型的方法是将标准的非私有算法视为黑盒，为它提供已经通过添加噪声来掩盖个人贡献而处理的输入，以便输出也必须保护隐私。该项目将打开这个黑匣子，产生新的算法，提高隐私保障，同时在理论和实践中保持良好的收敛结果。最初的技术方法之一将是在动量（现代优化算法中无处不在的技术）与隐私之间建立联系。这种方法不需要对损失面进行结构性假设（例如，该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Modern Artificial intelligence (AI) systems are typically built with the use of large datasets that may be generated by individuals contributing data over the internet, such as through product ratings, comments, or other online interactions. As a result, it is critical that such AI systems are able to preserve the privacy of the individuals whose data is used: it should not be possible for an outside observer to learn anything about any individual person through the use of the AI system. This project will investigate the fundamental limits of the trade-off^s between privacy and performance to build AI systems that are as performant as possible without compromising privacy. The project's results will not only improve the security of people who already benefit from products that learn from their data but will also reduce bias in AI by enabling more sensitive or vulnerable individuals to participate safely.On a technical level, this project will develop new differentially private stochastic optimization algorithms. In recent years, there has been a surge of interest in private optimization, but the theory for private non-convex optimization (which is required for training neural networks) is surprisingly underdeveloped. For this setting, the typical approach is to treat a standard non-private algorithm as a black box, providing it with inputs that have already been processed by adding noise to obscure individual contributions so that the output must also necessarily preserve privacy. This project will open up this black box to produce new algorithms that improve privacy guarantees while maintaining good convergence results both in theory and in practice. One of the initial technical approaches will be to develop a connection between momentum, a ubiquitous technique in modern optimization algorithms, with privacy. This approach does not require structural assumptions onthe loss surface (e.g., convexity, smoothness) to ensure privacy but has the potential to significantly decrease the amount of noise injected into the algorithm, resulting in improved performance with the same level of privacy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.