CIF: Small: A Novel Paradigm of Information Extraction in Big Data Problems

该项目考虑与超大样本量和超高维度的大数据量增加相关的问题。这些卷介绍了许多新的和独特的挑战，科学家在计算和统计推断。在大数据中发现和分离重要信息是一项极其艰巨的挑战。为了应对这些挑战，该项目将开发一个新的系统，可以提供大数据的质量分析。凭借研究者在理论，方法和应用统计研究方面的经验，该项目将为本科生和研究生提供一个极好的机会，让他们参与尖端的统计应用和方法开发，从而为他们未来的职业生涯做好准备。通过本地化数据，该项目开发了一个系统，该系统具有用于估计，计算，渐近研究和统计推断的新颖技术的连贯集合，克服了大数据的新挑战。该项目将在充分降维（SDR）和充分变量选择（SVS）方面带来新的研究方向，产生适用于广泛科学领域的新的大数据挖掘工具。这项工作是提高对SDR和SVS的理解的重要一步，包括它们的优点以及如何克服它们的缺点以适应大数据分析的挑战。该项目还将为科学家提供一个新的平台，以开发更灵活和有效的方法。研究人员将建立拟议的估计的理论属性，并开发新的可扩展的计算算法的超大规模和超高维的数据与一个开放访问的R包，以传播知识的科学界。这个奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

This project considers problems related to the increasing volume of big data characterized by ultra-large sample size and ultra-high dimensionality. Such volumes introduce many new and unique challenges to scientists in both computation and statistical inference. Finding and isolating the important information in big data is an extremely difficult challenge. In response to such challenges, the project will develop a novel system that can provide quality analysis of big data. With the investigator's experience in statistical research of theory, methodology, and applications, the project will provide an excellent opportunity for both undergraduate and graduate students to participate in cutting-edge statistical applications and methodology development, and thereby prepare them well for their future careers. By localizing the data, this project develops a system with a coherent collection of novel techniques for estimation, computation, asymptotic studies, and statistical inference overcoming the new challenges for big data. The project will lead to new research directions in sufficient dimension reduction (SDR) and sufficient variable selection (SVS), producing new big data mining tools applicable in a wide range of scientific fields. This work is a major step towards improving the understanding of SDR and SVS, including their advantages and how to overcome their disadvantages to suit for the challenge of big data analysis. This project will also provide scientists a new platform to develop more flexible and efficient methods. The investigator will establish theoretical properties of the proposed estimators and develop new scalable computation algorithms for ultra-size and ultra-high dimensional data with an open-access R package to disseminate the knowledge to the scientific community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.