Dimension Reduction for Non-Regular Statistical Models with Applications

该提案旨在发展关于高维非规则模型降维的新的统计理论和方法，这些模型允许相对于参数或协变量的子集不连续。这些模型自然产生于统计、生物统计、气候、营销研究、管理、经济和金融等各个领域的应用。它们可以捕捉数据结构的许多重要特征以及解释变量和响应变量之间的关联，这是低维或常规模型本身无法复制的。这一建议主要关注门限模型，这是一类重要的非规则模型，在统计学、生物统计学和经济学中有广泛的应用。虽然关于低维数据的门限模型的文献很多，但应用于高维数据的门限模型的统计理论和方法还没有发展起来，这是由于四个主要挑战：(I)估计的统计不规则性，(Ii)增加的维度，(Iii)响应变量的未知或不完全分布，(Iv)计算困难。通过介绍惩罚技术，提出了一些相关的研究课题以供研究。将开发用于大数据和高维数据的非规则模型的统计推断和计算算法的新工具，例如脑成像数据。这些新的发展将使科学家能够有效地分析数据，大幅提高灵活性、可解释性和减少建模偏差。此外，研究人员将把新的数学、概率和计算工具与科学和工程中的工具结合起来。这些进展的传播将促进新的知识发现，并加强跨学科合作。这项研究还将通过关于当代最先进的数据挖掘和机器学习的多学科课程达到教育目的，并有利于本科生、研究生和代表性不足的少数群体的培训和学习。

The proposal aims to develop new statistical theory and methodology on dimension reduction for high-dimensional non-regular models which allow for discontinuity with respect to a subset of the parameters or covariates. Such models arise naturally from applications in various fields, such as statistics, biostatistics, climate, marketing research, management, economics and finance. They can capture many important features of the data structure and association between the explanatory and response variables which either low-dimensional or regular models alone cannot duplicate. This proposal focuses primarily on threshold models, an important class of non-regular models which has a wide variety of applications in statistics, biostatistics, and economics. While the literature on threshold models for low-dimensional data is comprehensive, the statistical theory and methods for threshold models applied to high-dimensional data are undeveloped due to four central challenges: (I) statistical nonregularities of the estimation, (II) increasing dimensionality, (III) unknown or incomplete distributions of response variables, (IV) computational difficulties. By introducing penalization techniques, a number of related research topics are proposed for investigation. New tools for statistical inference and computational algorithms of non-regular models applied to large and high-dimensional data, for example the brain imaging data, will be developed.These new developments will allow scientists to efficiently analyze data with substantially increased flexibility, interpretability and reduced modeling biases. In addition, the investigator will integrate new mathematical, probabilistic and computational tools with those in sciences and engineering. Dissemination of these developments will enhance new knowledge discoveries, and strengthen interdisciplinary collaborations. The research will also serve an educational purpose through multi-disciplinary courses on the contemporary state-of-the-art data mining and machine learning, and benefit the training and learning of undergraduate, graduate students and underrepresented minorities.