CDS&E-MSS: Data thinning: methods, theory, and applications

近年来，几乎所有科学和工程领域的数据都出现了爆炸式增长。与此同时，分析这些数据的统计和机器学习方法的可用性和复杂性也在增加。然而，在验证所产生的模型方面出现了一个重大挑战，即确保它们捕获信号而不是噪音。本项目将开发一个新的框架，将数据集分为两部分，一部分用于拟合复杂的模型，另一部分用于验证模型。研究生将参与该项目，PI将努力实现统计学人才的多样化。 该项目探索了“数据细化”，这是一种将卷积封闭分布产生的数据集拆分为两个或多个独立分量的新思想，这些分量相加产生原始数据集。PI将把数据细化框架从卷积封闭分布推广到更广泛的分布类别，允许数据集被细化为两个或更多个独立的分量，这些分量可以使用充分性原则通过除加法之外的函数重新组合以产生原始数据集。一旦获得了两个或多个独立的组件，模型就可以拟合到一个组件，并在其他组件上进行验证。还将探讨与样本分割、数据裂变和其他相关建议的联系。PI还将与生物医学研究人员合作，根据来自不同领域的数据量身定制这一框架，并将发布高质量的软件。最终，数据细化框架将在模型拟合后实现严格的验证和推理，以解决科学和工程中的许多问题。更广泛的影响包括研究生参与这项研究，以及PI正在进行的多样性和包容性努力。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In recent years, there has been an explosion of data across virtually all areas of science and engineering. This has been paired with an increase in the availability and complexity of statistical and machine-learning methods to analyze these data. However, a major challenge arises in validating the resulting models, i.e. in making sure that they capture signal rather than noise. In this project, the project will develop a new framework to split up a dataset into two separate parts, so that one part can be used to fit a complex model, and the remaining part can be used to validate it. Graduate students will be involved in this project, and the PI will carry out efforts to diversify the workforce in statistics. This project explores “data thinning,” a new idea for splitting a dataset arising from a convolution-closed distribution into two or more independent components that sum to yield the original dataset. The PI will generalize the data thinning framework beyond convolution-closed distributions to a much broader class of distributions, by allowing for a dataset to be thinned into two or more independent components that can be re-combined to yield the original dataset via a function other than addition, using principles of sufficiency. Once two or more independent components have been obtained, a model can be fit to one component and validated on the other(s). Connections to sample splitting, data fission, and other related proposals will also be explored. The PI will also collaborate with biomedical researchers to tailor this framework to data from diverse domain areas and will release high-quality software. Ultimately, the data thinning framework will enable rigorous validation and inference after model fitting for a host of problems in science and engineering. Broader impacts include the involvement of graduate students in this research, as well as ongoing diversity and inclusion efforts by the PI.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.