SEMIPARAMETRIC INFERENCE WITH HIGH-DIMENSIONAL DATA

由于信息技术的快速发展及其在现代科学实验中的应用，大数据是当前统计研究和实践中一个非常感兴趣的领域。高维统计方法通常为复杂大数据问题的工程解决方案提供关键要素和思路。具有大量此类问题的重要领域包括生物信息学、信号处理、神经成像、通信和社交网络、文本挖掘等。在许多这样的应用中，问题的名义复杂性，通常由数据的维度来衡量，如生物信息学中的遗传成分，神经成像中的大脑区域或体素，或互联网中的计算机和路由器，远远大于样本点的数量或数据的信息内容。该研究项目将确定高维统计模型和可行有效统计推断的问题，并将开发新的方法和算法，以便用高维数据进行这种有效的统计推断。拟议的研究是由现代信息技术繁荣的上述领域的现实生活问题所驱动的，并将直接适用于这些问题。此外，拟议的研究将具有重大的教育影响。在高维数据中，一个长期存在的挑战是在不依赖于模型选择一致性理论的情况下识别常规统计推断可行的问题。一致的模型选择允许通过识别所有相关特征将问题的名义复杂性降低到可管理的水平。然而，模型选择一致性通常需要统一的强信号来分离相关特征和不相关特征。不幸的是，这种均匀的信号强度假设很少得到数据或基础科学的支持，特别是在生物学、医学和社会学应用中。PI提出了一种半低维的统计推断方法，并成功地将其应用于构造高维回归和图形模型中的规则p值和置信区间。这种方法纠正了模型选择器的偏差，就像半参数方法纠正了非参数估计器的偏差一样。这项提议的研究将进一步发展这种方法在高维数据分析中的应用，并以几年前不可见的方式解决新问题。它将侧重于半监督数据的有效统计推断和涉及许多高维或复杂成分的问题，包括高维数据的复合和多元特征的置信区域和重要测试。该项目将开发实用的方法、高效的算法、统计软件和与涉及许多高维或复杂组件的常见应用直接相关的坚实理论。

Big Data is an area of intense current interest in statistical research and practice due to the rapid development of information technologies and their applications to modern scientific experiments. High-dimensional statistical methods typically provide crucial elements and ideas in engineering solutions for complex Big Data problems. Important fields with an abundance of such problems include bioinformatics, signal processing, neural imaging, communications and social networks, text mining and more. In many such applications, the nominal complexity of the problem, typically measured by the dimension of the data such as genetic components in bioinformatics, brain regions or voxels in neural imaging, or computers and routers in the Internet, is much greater than number of sample points or the information content of the data. The research project will identify and characterize high-dimensional statistical models and problems in which efficient statistical inference are feasible, and will develop new methodologies and algorithms to carry out such efficient statistical inference with high-dimensional data. The proposed research is motivated by and will be directly applicable to real life problems in the aforementioned areas where modern information technologies prosper. Furthermore, the proposed research will have significant educational impact. A longstanding challenge in high-dimensional data is to identify problems where regular statistical inference is feasible without relying on model selection consistency theory. Consistent model selection allows reduction of the nominal complexity of the problem to a manageable level by identifying all relevant features. However, model selection consistency typically requires uniformly strong signal to separate relevant features from irrelevant ones. Unfortunately, such uniform signal strength assumption is seldom supported by either the data or the underlying science, especially in biological, medical and sociological applications. The PI has proposed a semi-low-dimensional approach of statistical inference and successfully applied it to construct regular p-values and confidence intervals in high-dimensional regression and graphical models. This approach corrects the bias of model selectors just as semiparametric approach corrects the bias of nonparametric estimators. The proposed research will further develop this approach in high-dimensional data analysis and tackle new problems in ways not visible just a few years ago. It will focus on efficient statistical inference with semisupervised data and problems involving many high-dimensional or complex components, including confidence regions and significant tests for composite and multivariate features with high-dimensional data. The project will develop practical methods, efficient algorithms, statistical software, and solid theory directly relevant to common applications involving many high-dimensional or complex components.