RI: Small: Learning discrete structure from continuous spaces

RI：小：从连续空间学习离散结构

• 批准号: 2050360
• 负责人: Mikhail Belkin
• 金额: $ 27.08万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-10 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2050360&HistoricalAwards=false
• 关键词: RI; Small; Learning; discrete; structure

Science, medicine, business, and engineering are increasingly data-driven. Hypotheses, diagnoses, decisions, and designs are made by gathering and analyzing a wealth of data in search of meaningful patterns. It is the goal of the field of machine learning to develop methods for inferring and reasoning about patterns in the data. The research supported by this award addresses the question of learning from unlabeled data, one of the more vexing problems of data science. The PI's analyze and develop algorithms for problems such as clustering, i.e., finding groups of similar objects, as well as understanding continuously changing attributes in data. By integrating machine learning, modeling and geometric data analysis, this work injects new ideas and methodologies to modern data analysis, helps build practical algorithms for unsupervised and unsupervised learning and analyze their properties and domains of applicability. Students working on this project have a unique opportunity to be exposed to a broad spectrum of topics including machine learning, statistics, geometry and applied mathematics.On a more technical level, the unifying perspective for the proposed research is that many of these unsupervised learning problems can be viewed as recovering structure or invariants of the underlying continuous space through the lens of the discrete data. This work takes that point of view to consider a number of important aspects of unsupervised learning including hierarchical clustering in the density model, data quantization, graphon clustering and estimation, as well as learning metric structure from data. The project also considers applications of these ideas to supervised learning, particularly in helping to scale algorithms to large data. While the work on this project concentrates on theoretical analyses, these are developed with a view toward practical algorithms, implementations and applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

科学、医学、商业和工程越来越受数据驱动。假设、诊断、决策和设计都是通过收集和分析大量数据来寻找有意义的模式。机器学习领域的目标是开发用于推断和推理数据中的模式的方法。该奖项支持的研究解决了从未标记数据中学习的问题，这是数据科学中最令人烦恼的问题之一。 PI的分析和开发算法的问题，如聚类，即，查找相似对象的组，以及理解数据中不断变化的属性。通过整合机器学习，建模和几何数据分析，这项工作为现代数据分析注入了新的思想和方法，有助于构建无监督和无监督学习的实用算法，并分析其属性和适用范围。参与该项目的学生有一个独特的机会接触到广泛的主题，包括机器学习，统计，几何和应用数学。在更技术的层面上，拟议研究的统一观点是，许多这些无监督学习问题可以被视为通过离散数据的透镜恢复底层连续空间的结构或不变量。这项工作从这个角度考虑了无监督学习的一些重要方面，包括密度模型中的层次聚类，数据量化，graphon聚类和估计，以及从数据中学习度量结构。该项目还考虑将这些想法应用于监督学习，特别是帮助将算法扩展到大数据。 虽然该项目的工作集中在理论分析上，但这些都是为了实际的算法，实现和应用而开发的。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks

• 发表时间: 2020-06
• 期刊: ArXiv
• 作者: Like Hui;M. Belkin

On the linearity of large non-linear models: when and why the tangent kernel is constant

• 发表时间: 2020-10
• 期刊: ArXiv
• 作者: Chaoyue Liu;Libin Zhu;M. Belkin

Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation

• DOI: 10.1017/s0962492921000039
• 发表时间: 2021-05-01
• 期刊: ACTA NUMERICA
• 影响因子: 14.2
• 作者: Belkin, Mikhail

Loss landscapes and optimization in over-parameterized non-linear systems and neural networks

• DOI: 10.1016/j.acha.2021.12.009
• 发表时间: 2022-04-25
• 期刊: APPLIED AND COMPUTATIONAL HARMONIC ANALYSIS
• 影响因子: 2.5
• 作者: Liu, Chaoyue;Zhu, Libin;Belkin, Mikhail
• 通讯作者: Belkin, Mikhail

On the Inconsistency of Kernel Ridgeless Regression in Fixed Dimensions

固定维数下核无脊回归的不一致性

• DOI: 10.1137/22m1499819
• 发表时间: 2023
• 期刊: SIAM Journal on Mathematics of Data Science
• 影响因子: 3.6
• 作者: Beaglehole, Daniel;Belkin, Mikhail;Pandit, Parthe
• 通讯作者: Pandit, Parthe