SEMIPARAMETRIC INFERENCE WITH HIGH-DIMENSIONAL DATA

由于信息技术的快速发展及其在现代科学实验中的应用，大数据是当前统计研究和实践中的一个热门领域。高维统计方法通常为复杂的大数据问题的工程解决方案提供关键元素和想法。具有大量此类问题的重要领域包括生物信息学、信号处理、神经成像、通信和社交网络、文本挖掘等。在许多这样的应用中，问题的标称复杂性通常由数据的维度（诸如生物信息学中的遗传成分、神经成像中的大脑区域或体素、或互联网中的计算机和路由器）来测量，远大于样本点的数量或数据的信息内容。该研究项目将确定和表征高维统计模型和问题，其中有效的统计推断是可行的，并将开发新的方法和算法来进行这种有效的统计推断与高维数据。拟议的研究的动机，并将直接适用于真实的生活中的问题，在上述领域的现代信息技术的繁荣。此外，拟议的研究将产生重大的教育影响。高维数据中的一个长期挑战是识别规则统计推断可行的问题，而不依赖于模型选择一致性理论。一致的模型选择允许通过识别所有相关特征将问题的标称复杂性降低到可管理的水平。然而，模型选择的一致性通常需要一致的强信号来区分相关特征和不相关特征。不幸的是，这种均匀的信号强度假设很少得到数据或基础科学的支持，特别是在生物学，医学和社会学应用中。PI提出了一种统计推断的半低维方法，并成功地将其应用于高维回归和图形模型中的常规p值和置信区间的构建。这种方法纠正模型选择的偏差，就像半参数方法纠正非参数估计的偏差一样。拟议的研究将进一步发展这种方法在高维数据分析和解决新的问题的方式还没有看到几年前。它将专注于半监督数据的有效统计推断和涉及许多高维或复杂组件的问题，包括置信区域和高维数据的复合和多变量特征的显著性检验。该项目将开发实用的方法，高效的算法，统计软件和坚实的理论直接相关的常见应用程序涉及许多高维或复杂的组件。

Big Data is an area of intense current interest in statistical research and practice due to the rapid development of information technologies and their applications to modern scientific experiments. High-dimensional statistical methods typically provide crucial elements and ideas in engineering solutions for complex Big Data problems. Important fields with an abundance of such problems include bioinformatics, signal processing, neural imaging, communications and social networks, text mining and more. In many such applications, the nominal complexity of the problem, typically measured by the dimension of the data such as genetic components in bioinformatics, brain regions or voxels in neural imaging, or computers and routers in the Internet, is much greater than number of sample points or the information content of the data. The research project will identify and characterize high-dimensional statistical models and problems in which efficient statistical inference are feasible, and will develop new methodologies and algorithms to carry out such efficient statistical inference with high-dimensional data. The proposed research is motivated by and will be directly applicable to real life problems in the aforementioned areas where modern information technologies prosper. Furthermore, the proposed research will have significant educational impact. A longstanding challenge in high-dimensional data is to identify problems where regular statistical inference is feasible without relying on model selection consistency theory. Consistent model selection allows reduction of the nominal complexity of the problem to a manageable level by identifying all relevant features. However, model selection consistency typically requires uniformly strong signal to separate relevant features from irrelevant ones. Unfortunately, such uniform signal strength assumption is seldom supported by either the data or the underlying science, especially in biological, medical and sociological applications. The PI has proposed a semi-low-dimensional approach of statistical inference and successfully applied it to construct regular p-values and confidence intervals in high-dimensional regression and graphical models. This approach corrects the bias of model selectors just as semiparametric approach corrects the bias of nonparametric estimators. The proposed research will further develop this approach in high-dimensional data analysis and tackle new problems in ways not visible just a few years ago. It will focus on efficient statistical inference with semisupervised data and problems involving many high-dimensional or complex components, including confidence regions and significant tests for composite and multivariate features with high-dimensional data. The project will develop practical methods, efficient algorithms, statistical software, and solid theory directly relevant to common applications involving many high-dimensional or complex components.