New Studies of Learning with Stochastic Convex Optimization

从数据中学习的范式在科学和工程中发挥着越来越重要的作用。 机器学习和数学优化之间的相互作用是最富有成效的，其中一个突出的领域是随机凸优化（SCO）。然而，对于推广和稳定性分析等基本问题的研究相对较少，现有的研究往往集中在光滑损失下的标准分类和回归上。此外，收集和用于学习的数据通常包含敏感信息，例如欺诈检测的财务记录或癌症诊断的基因组数据，这迫切需要开发具有理论保证的隐私保护SCO算法。这些为该项目提供了动力，该项目旨在研究机器学习启发的SCO算法的基本特性，包括其稳定性，泛化和差分隐私。学生将参与并接受跨学科方面的培训。拟议工作的技术目标分为三个重点。 第一个重点是研究随机梯度方法（SGM）解决与非光滑损失相关的SCO问题的稳定性和推广。第二个重点是开发和研究SGM算法的SCO问题，可以防止隐私泄漏使用一个公认的数学定义的隐私称为差分隐私。第三个重点是研究SCO算法的稳定性、泛化性和差分隐私，用于比标准分类和回归更复杂的损失的成对学习。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The paradigm of learning from data is playing an increasingly important role in science and engineering. The interplay between machine learning and mathematical optimization has been most fruitful, and one prominent area is stochastic convex optimization (SCO). However, there is relatively little work on the fundamental questions such as generalization and stability analysis, and the existing studies often focus on the standard classification and regression with smooth losses. Furthermore, data collected and used for the learning often contains sensitive information such as financial records from fraud detection or genomic data from cancer diagnosis which presents an urgent need to develop privacy-preserving SCO algorithms with theoretical guarantees. These provide motivation for the project which aims to study the fundamental properties of machine learning inspired SCO algorithms including their stability, generalization, and differential privacy. Students will be involved and trained in interdisciplinary aspects. The technical objectives of the proposed work are divided into three thrusts. The first thrust focuses on the study of stability and generalization of stochastic gradient methods (SGM) for solving SCO problems associated with non-smooth losses. The second thrust is to develop and study SGM algorithms for SCO problems which can prevent the privacy leakage using a well-accepted mathematical definition of privacy called differential privacy. The third thrust is to study the stability, generalization, and differential privacy of SCO algorithms for pairwise learning which involves more complex losses than the standard classification and regression.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.