Collaborative Research: Information-Based Subdata Selection Inspired by Optimal Design of Experiments

协作研究：受实验优化设计启发的基于信息的子数据选择

• 批准号: 1811291
• 负责人: Min Yang
• 金额: $ 6万
• 依托单位: University of Illinois at Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-15 至 2022-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1811291&HistoricalAwards=false
• 关键词: Collaborative; Research; Information; Based; Subdata

Extraordinary amounts of data are collected in many branches of science, in industry, and in government. The massive amounts of data provide incredible opportunities for making knowledge-based decisions and for advancing complicated research problems through data-driven discoveries. To capitalize on these opportunities, it is critical to develop methodology that facilitates the extraction of useful information from massive data in a computationally efficient way. Even the simplest analyses of the data can be computationally intensive or may no longer be feasible for big data. It is however often the case that valid conclusions can be drawn by considering only some of the data, referred to as subdata. This project develops optimal strategies for selecting subdata that retain, as much as possible, relevant information that was available in the massive data set. The methodology helps to identify the most informative data points, after which an analysis can proceed based on the selected subdata only. This facilitates data-driven decisions, scientific discoveries, and technological breakthroughs with computing resources that are readily available. Existing investigations for extracting information from big data with common computing power have focused on random subsampling-based approaches, which have as limitation that the amount of information extracted is only scalable to the subdata size, not the full data size. This project develops and expands the Information-Based Optimal Subdata Selection (IBOSS) method proposed by the PIs in the following directions: 1) It combines IBOSS with sparse variable selection methods in linear regression; 2) it develops subdata selection methods for generalized linear models; 3) it constructs computationally efficient algorithms for selecting the most informative subdata; and 4) it develops user-friendly software that supports the methodology. The research is a significant addition to the field of big data science. It advances a new method for dealing with big data and has the potential to create novel research opportunities in statistical science and other quantitative fields. The results are valuable even when supercomputers are available, because cutting edge high performance computing facilities will always trail the exponential growth of data volume.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在科学的许多分支、工业和政府中收集了大量的数据。大量的数据为做出基于知识的决策和通过数据驱动的发现推进复杂的研究问题提供了令人难以置信的机会。为了利用这些机会，关键是要开发一种方法，以便于以计算效率高的方式从海量数据中提取有用的信息。即使是最简单的数据分析也可能是计算密集型的，或者可能不再适用于大数据。然而，通常情况下，只考虑部分数据（称为子数据）就可以得出有效的结论。该项目开发了选择子数据的最佳策略，这些子数据尽可能多地保留了海量数据集中可用的相关信息。该方法有助于确定信息量最大的数据点，然后仅根据选定的子数据进行分析。这有助于数据驱动的决策，科学发现和技术突破，以及随时可用的计算资源。现有的从具有普通计算能力的大数据中提取信息的研究集中在基于随机子采样的方法上，其局限性在于提取的信息量仅可扩展到子数据大小，而不是完整的数据大小。本计画发展及扩充由PI所提出的基于信息的最佳子数据选择（Information-Based Optimal Subdata Selection，IBOSS）方法，其方向如下：1）将IBOSS与线性回归中的稀疏变量选择方法结合; 2）发展广义线性模型的子数据选择方法; 3）建构计算效率高的算法，以选择最具信息量的子数据;以及4）它开发了支持该方法的用户友好软件。 这项研究是对大数据科学领域的重要补充。它提出了一种处理大数据的新方法，并有可能在统计科学和其他定量领域创造新的研究机会。即使在超级计算机可用的情况下，结果也是有价值的，因为尖端的高性能计算设施总是会跟踪数据量的指数增长。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Information-based optimal subdata selection for big data logistic regression

• DOI: 10.1016/j.jspi.2020.03.004
• 发表时间: 2020-12
• 期刊: Journal of Statistical Planning and Inference
• 影响因子: 0.9
• 作者: Q. Cheng;Haiying Wang;Min Yang

Optimal design under complete class with ancillary functions

全类下优化设计，附带辅助功能

• DOI: 10.1002/cjs.11596
• 发表时间: 2021
• 期刊: The Canadian journal of statistics
• 作者: Hua, Y. and

Support point of locally optimal designs for multinomial logistic regression models

多项逻辑回归模型局部最优设计的支撑点

• DOI: 10.1016/j.jspi.2020.03.006
• 发表时间: 2020
• 期刊: Journal of Statistical Planning and Inference
• 影响因子: 0.9
• 作者: Hao, Shuai;Yang, Min
• 通讯作者: Yang, Min

On multiple-objective optimal designs

• DOI: 10.1016/j.jspi.2018.09.007
• 发表时间: 2019-05-01
• 期刊: JOURNAL OF STATISTICAL PLANNING AND INFERENCE
• 影响因子: 0.9
• 作者: Cheng, Qianshun;Yang, Min
• 通讯作者: Yang, Min