Applications of variational analysis in optimization and data science

变分分析在优化和数据科学中的应用

• 批准号: RGPIN-2017-04035
• 负责人: Hoheisel, Tim
• 金额: $ 1.38万
• 依托单位: McGill University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=710657
• 关键词: Applications; variational; analysis; optimization; data

An abundance of statistical methods and learning algorithms reduce to an optimization problem. With the advent of computers and the information age these problems have exploded both in size and complexity. One of the most important questions nowadays is how to extract trends and substructures in extremely large data sets. In this area of data science, which comprises data mining, machine learning, support vector machines or signal processing, techniques like sparse and low-rank optimization or compressed sensing are frequently used. A common feature of the resulting optimization problems is nonsmoothness of the occurring functions. Hence, there is an increased demand for nonsmooth optimization methods. These rely heavily on solid mathematical foundations in nonsmooth and set-valued analysis. Therefore, as a long-term goal, this research program aims at developing novel variational tools and bring them to bear on solution methods for nonsmooth optimization problems occurring in a variety of fields such as data science and learning. A watershed in optimization, in particular in the nonsmooth setting, is convexity. This is due to the fact that convex functions exhibit many desirable properties: They allow for a powerful subdifferential and duality calculus. Moreover, stationary points, local and global minima coincide and they have desirable analytical features such as affine minorization, (local) Lipschitz properties or prox-regularity. Although many problems in practice are not fully convex, there are often convex substructures which can (and should) be exploited. We take this as a guideline for this program in which we focus on two intimately related topics in nonsmooth optimization which have strong connections to statistical and machine learning as well as to other current areas in optimization. Primarily, we would like to study DC optimization, i.e. minimization problems where the objective function is the difference of two convex functions. This well-established nonconvex problem class covers an abundance of applications and many of the problems of our interest. We also lay a focus on the variational analysis of some of the concrete nonsmooth, convex functions occurring in various applications such as the matrix-fractional function, which seems to be ubiquitous in the area of data science and connects different topics such as quadratic optimization, multitask learning and nuclear norm smoothing.

大量的统计方法和学习算法归结为优化问题。 随着计算机和信息时代的到来，这些问题的规模和复杂性都急剧增加。如今最重要的问题之一是如何在超大数据集中提取趋势和子结构。在这个数据科学领域，包括数据挖掘，机器学习，支持向量机或信号处理，经常使用稀疏和低秩优化或压缩感知等技术。 由此产生的优化问题的一个共同特征是出现的功能的非光滑性。因此，对非光滑优化方法的需求增加。这些在很大程度上依赖于非光滑和集值分析的坚实数学基础。 因此，作为一个长期的目标，本研究计划旨在开发新的变分工具，并使他们承担 求解方法 非光滑优化问题发生在各种领域，如数据科学和学习。 凸性是最优化中的一个分水岭，特别是在非光滑的情况下。这是因为凸函数表现出许多理想的性质：它们允许强大的次微分和对偶演算。 此外，稳定点，局部和全局最小值重合，他们有理想的分析功能，如仿射minorization，（局部）Lipschitz性质或正则性。 虽然许多问题在实践中并不是完全凸的，但通常有凸的子结构可以（也应该）被利用。我们把这作为这个节目的指导方针，我们把重点放在两个密切相关的 非光滑优化专题 它与统计和机器学习以及其他当前优化领域有着密切的联系。 首先，我们想研究DC优化，即目标函数是两个凸函数之差的最小化问题。这类完善的非凸问题涵盖了大量的应用和我们感兴趣的许多问题。 我们还关注于各种应用中出现的一些具体的非光滑凸函数的变分分析，例如矩阵分数函数，它似乎在数据科学领域无处不在，并连接了不同的主题，如二次优化，多任务学习和核范数平滑。