CAREER: Learning and Selecting Low-Dimensional Models from Incomplete Data

职业：从不完整数据中学习和选择低维模型

• 批准号: 2239479
• 负责人: Daniel Pimentel-Alarcon
• 金额: $ 60万
• 依托单位: University of Wisconsin-Madison
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-02-01 至 2028-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2239479&HistoricalAwards=false
• 关键词: CAREER; Learning; Selecting; Low; Dimensional

Big datasets often have an underlying structure. Identifying such a structure allows predicting outcomes of interest based on a few variables, for example, predicting the effectiveness of a drug or vaccine based on the drug’s molecular structure. There exists a wide variety of methods to learn the underlying structure of a dataset and make accurate predictions. However, when data is severely incomplete, as is the case in many modern datasets, existing methods consistently fail to identify the correct structure of the data. More alarmingly, the existing methodology has no means to verify whether the structure found is correct or not. In other words, whenever data is incomplete, the structure learned by any existing method cannot be trusted and may result in undetectable, arbitrarily wrong predictions. This project will (i) develop methods to learn structures specifically tailored to handle missing data and (ii) develop a theory to verify whether the structure learned by any method (including existing ones) is correct or not. In turn, this research will enable scientists to learn the structures governing their incomplete datasets in a plethora of applications to the benefit of society, including drug discovery, metagenomics, and opportunistic screening. Furthermore, this project will support outreach activities to engage underrepresented minorities in machine learning, both locally and nationally, through hands-on activities, social media campaigns, symposia, courses, and mentoring.The technical aims of the project are divided into three main thrusts. The first thrust will investigate a new approach that maps incomplete data to the Grassmann manifold of subspaces, wherein the data’s underlying structure can be revealed by solving a constrained optimization over the Schubert varieties defined by the observed data. The second thrust will develop model-selection criteria to determine the structure that best fits an incomplete dataset, among a collection of candidate structures. These criteria will be generalizations of the Akaike and Bayes information criteria and the minimum effective dimension, adapted to account for missing data. These criteria will be complemented with a goodness-of-fit test to determine if the winning structure is, indeed, a good fit for the data. These are non-trivial tasks that require special considerations in light of missing data, which can consistently cause spurious structures fit arbitrarily large datasets with the same degree of error as the correct structures. Ultimately, the results from this thrust will allow determining whether the predictions stemming from a specific structure can be trusted or not. The third thrust will implement our methodology in open-source, easy-to-use software to benefit of the broader scientific community and test it on datasets related to our ongoing interdisciplinary collaborations in metagenomics, single-cell sequencing, sonotypes classification, bacteria classification, drug discovery, and clinical opportunistic screening.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

大数据集通常具有底层结构。识别这种结构可以根据一些变量预测感兴趣的结果，例如根据药物的分子结构预测药物或疫苗的有效性。有多种方法可以学习数据集的底层结构并做出准确的预测。然而，当数据严重不完整时，就像许多现代数据集中的情况一样，现有方法始终无法识别数据的正确结构。更令人担忧的是，现有的方法无法验证所发现的结构是否正确。换句话说，只要数据不完整，任何现有方法学习到的结构都不可信，并且可能导致无法检测的、任意错误的预测。该项目将（i）开发学习专门用于处理缺失数据的结构的方法，以及（ii）开发一种理论来验证通过任何方法（包括现有方法）学习的结构是否正确。反过来，这项研究将使科学家能够在众多造福社会的应用中了解管理不完整数据集的结构，包括药物发现、宏基因组学和机会性筛选。此外，该项目还将支持外展活动，通过实践活动、社交媒体活动、研讨会、课程和指导，让当地和全国范围内代表性不足的少数群体参与机器学习。该项目的技术目标分为三个主要目标。第一个主旨将研究一种将不完整数据映射到格拉斯曼流形子空间的新方法，其中可以通过解决由观测数据定义的舒伯特簇的约束优化来揭示数据的底层结构。第二个重点将制定模型选择标准，以确定在候选结构集合中最适合不完整数据集的结构。这些标准将是 Akaike 和贝叶斯信息标准和最小有效维度的概括，适用于解决缺失数据。这些标准将辅以拟合优度检验，以确定获胜结构是否确实适合数据。这些都是重要的任务，需要根据丢失的数据进行特殊考虑，这可能始终导致虚假结构适合任意大的数据集，并且与正确结构具有相同程度的错误。最终，这一推力的结果将有助于确定源自特定结构的预测是否可信。第三个重点将在开源、易于使用的软件中实施我们的方法，以使更广泛的科学界受益，并在与我们在宏基因组学、单细胞测序、声型分类、细菌分类、药物发现和临床机会筛选方面正在进行的跨学科合作相关的数据集上进行测试。该奖项反映了 NSF 的法定使命，并通过使用基金会的评估进行评估，被认为值得支持。 智力价值和更广泛的影响审查标准。