CIF: Small: A Novel Paradigm of Information Extraction in Big Data Problems

该项目考虑了大数据数据量不断增加的相关问题，这些数据具有超大样本容量和超高维的特点。这样的体积给科学家在计算和统计推断方面带来了许多新的和独特的挑战。找到并隔离大数据中的重要信息是一项极其困难的挑战。为了应对这些挑战，该项目将开发一种新的系统，可以提供大数据的质量分析。凭借研究人员在理论、方法和应用方面的统计研究经验，该项目将为本科生和研究生提供一个参与尖端统计应用和方法开发的绝佳机会，从而为他们未来的职业生涯做好准备。通过本地化数据，该项目开发了一个系统，该系统具有一套连贯的新技术，用于估计、计算、渐近研究和统计推断，以克服大数据面临的新挑战。该项目将在充分降维(SDR)和充分变量选择(SVS)方面带来新的研究方向，产生适用于广泛科学领域的新的大数据挖掘工具。这项工作是朝着提高对SDR和SVS的理解迈出的重要一步，包括它们的优势以及如何克服它们的缺点以适应大数据分析的挑战。该项目还将为科学家提供一个新的平台，以开发更灵活和高效的方法。研究人员将建立建议估计器的理论性质，并利用开放获取的R包为超大小和超高维数据开发新的可伸缩计算算法，以向科学界传播知识。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project considers problems related to the increasing volume of big data characterized by ultra-large sample size and ultra-high dimensionality. Such volumes introduce many new and unique challenges to scientists in both computation and statistical inference. Finding and isolating the important information in big data is an extremely difficult challenge. In response to such challenges, the project will develop a novel system that can provide quality analysis of big data. With the investigator's experience in statistical research of theory, methodology, and applications, the project will provide an excellent opportunity for both undergraduate and graduate students to participate in cutting-edge statistical applications and methodology development, and thereby prepare them well for their future careers. By localizing the data, this project develops a system with a coherent collection of novel techniques for estimation, computation, asymptotic studies, and statistical inference overcoming the new challenges for big data. The project will lead to new research directions in sufficient dimension reduction (SDR) and sufficient variable selection (SVS), producing new big data mining tools applicable in a wide range of scientific fields. This work is a major step towards improving the understanding of SDR and SVS, including their advantages and how to overcome their disadvantages to suit for the challenge of big data analysis. This project will also provide scientists a new platform to develop more flexible and efficient methods. The investigator will establish theoretical properties of the proposed estimators and develop new scalable computation algorithms for ultra-size and ultra-high dimensional data with an open-access R package to disseminate the knowledge to the scientific community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.