Statistical Inferences on Massive Data

该提案计划开发新的统计理论和方法来处理大量数据。 四个相互关联的途径提出了理论研究和方法的发展：高维变量选择，大协方差估计，大规模假设检验，非参数统计学习。特别是，提出了新的统计技术来回答以下重要问题：如何用一些获得的知识筛选基因和风险因素，折叠凹惩罚方法的优点是什么，如何估计分类和回归的基准，如何处理离群值，依赖数据和内源性测量，如何使用地理邻域的同质性来增强预测和推断，如何评估风险度量的不确定性，如何进行稀疏主成分分析，如何控制任意依赖下的错误发现率，如何使用非参数方法增强高维统计学习的灵活性。 此外，一个新的统计模型，从金融经济学理论的动机，提出了估计大的协方差矩阵，更好地理解风险的相关性，更好地评估风险。 检验内生变量和著名的多因素定价模型的存在的方法也将被提出，并进行深入研究。在探索科学前沿的过程中，大量数据收集已成为惯例，在一个案例中是基因组研究，在另一个案例中是衡量经济风险。 拟议的研究将增进我们对分子机制、生物过程、遗传关联、大脑功能、社交网络、经济和金融风险、供求关系的了解，从而提高经济和全球竞争力。此外，所提出的新的统计技术可以应用于其他生物和工程问题。该项目将通过与高年级本科生，研究生和博士后研究员密切合作来整合研究和教育，并通过与工业合作伙伴密切合作，开发公开可用的计算机代码来处理具有良好理论支持的海量数据，以增加学术界和工业界之间的合作。研究结果将通过在研讨会、会议、专业协会会议和互联网上的介绍广泛传播。

The proposal plans to develop novel statistical theory and methods for processing massive data. Four interrelated avenues are proposed for theoretical research and methodological developments: High-dimensional variable selection, large covariance estimation, large-scale hypothesis testing, and nonparametric statistical learning. In particular, novel statistical techniques are proposed to answer the following important questions: how to screen genes and risk factors with some acquired knowledge, what are the advantages of folded concave penalized methods, how to estimate the benchmark for classifications and regressions, how to deal with outliers, dependence data, and endogenous measurements, how to use homogeneity of geographical neighborhoods to enhance forecasting and inferences, how to assess uncertainty of risk measurements, how to conduct sparse principal component analysis, how to control the false discovery rates under arbitrary dependence, how to use nonparametric methods to enhance the flexibility of high-dimensional statistical learning. In addition, a novel statistical model, motivated from a financial economics theory, is proposed for estimating large covariance matrices for better understanding risk correlations and for better assessment of risks. The methods for testing the presence of endogenous variables and the celebrated multifactor pricing models are also presented and will be thoroughly investigated. Massive data collections have become routine in exploring the frontiers of science, in one case genomic studies and in another case measuring economic risks. The proposed research will advance our knowledge on understanding molecular mechanisms, biological processes, genetic associations, brain functions, social networks, economic and financial risks, supply and demands, and hence increase economic and global competitiveness. In addition, the proposed novel statistical techniques can be applied to other biological and engineering problems. The project will integrate research and education by working closely with senior undergraduate students, graduate students and postdoctoral fellows, and increase the collaborations between academia and industry by working closely with industrial partners and developing publicly available computer code for processing massive data with sound theoretical supports. The results will be disseminated broadly through presentations at seminars, conferences, professional association meetings, and the internet.