Flexible Modeling for High-Dimensional Complex Data: Theory, Methodology, and Computation

在高维数据分析中，预测因子之间的关系可能是高度非线性和非可加性的，考虑到这种复杂的结构可以显著提高模型的预测能力，并为了解潜在的数据生成机制提供重要的见解。该项目的目标是开发和研究新的统计和数据挖掘方法，用于检测高维稀疏模型中的非线性和非加性模式。当数据维数很高时，由于维数的诅咒，交互选择在数值和理论上都是极具挑战性的。实践中可用的工具非常有限，理论也很缺乏。在本项目中，研究者对高维相互作用选择问题进行了全面的研究。他们提出并研究了各种回归和分类模型的新选择和建模技术。推导出快速、鲁棒的大规模计算算法。此外，研究者还致力于建立相互作用选择的高维理论，为新方法提供坚实的基础。研究人员还提出并研究了一个统一的理论和计算框架，以识别一类广泛的非参数回归模型的非线性效应。特别的努力花在解决计算问题上，如多参数调优，正则化解决路径/表面算法，以及开发用户友好的统计软件包。大数据和高维数据为我们提供了从数据中获得非凡洞察力的迷人和前所未有的机会。另一方面，数据的规模和数量给标准分析工具提取有用信息带来了巨大的挑战。该项目的目标是开发创新的统计和数据挖掘方法、坚实的数学理论以及强大的计算工具和软件，以便在数据维数较高时捕获隐藏的和可能复杂的模式。在这个项目中需要解决的一个具有挑战性的问题是高维交互选择。在全基因组关联研究（GWAS）中，越来越多的证据表明，基因-基因和基因-环境相互作用可以为了解支撑人类疾病的复杂生物学途径提供关键见解。然而，在实践中，很少有有效的、基础良好的、计算上有吸引力的工具可用于识别高维数据的相互作用。调查人员试图通过对问题进行彻底调查来填补这一空白。该项目的研究结果可以显著推进理论，并为实际应用提供新的统计工具。所提出的方法具有广泛的科学应用，如生物学，生物医学和环境研究。该项目通过开发新的研究生和本科课程以及让学生参与研究活动，将研究、教育和跨学科合作结合起来。

In high dimensional data analysis, the relationships among predictors can be highly nonlinear and non-additive, and taking into account such complex structures may significantly improve model prediction power and provide crucial insight about the underlying data generation mechanism. The goal of this project is to develop and study new statistical and data mining methodologies for detecting nonlinear and non-additive patterns in high dimensional sparse models. When the data dimension is ultra-high, interaction selection is extremely challenging, both numerically and theoretically, due to curse of dimensionality. There are very limited tools available in practice and theory is scant. In this project, the investigators give a comprehensive treatment to the problem of high-dimensional interaction selection. They propose and study novel selection and modeling techniques for a variety of regression and classification models. Fast and robust large-scale computational algorithms are derived. In addition, the investigators are committed to establishing high dimensional theory for interaction selection and providing a solid foundation for the new methods. The investigators also propose and study a unified theory and computation framework to identify nonlinear effects for a broad class of nonparametric regression models. Special effort is spent on addressing computational issues such as multiple parameter tuning, regularization solution path/surface algorithms, and development of user friendly statistical software packages.Big and high dimensional data offer us fascinating and unprecedented opportunities to gain extraordinary insight from data. On the other hand, the scale and volume of data create tremendous challenges for standard analysis tools to extract useful information. The goal of this project is to develop innovative statistical and data mining methods, solid mathematical theory, and powerful computational tools and software to capture hidden and possibly complex patterns when the data dimension is high. One challenging problem to be tackled in this project is high dimensional interaction selection. In genome-wide association studies (GWAS), there is growing evidence that gene-gene and gene-environment interactions can provide key insight about complex biological pathways that underpin human diseases. However, there are very few effective, well-grounded, and computationally attractive tools available in practice to identify interactions for high dimensional data. The investigators try to fill this gap by conducting thorough investigation on the problem. The results from this project research can significantly advance theory and as well as contribute new statistical tools for practical use. The proposed methods have a wide range of scientific applications such as biology, biomedicine, and environmental studies. This project integrates research, education, and interdisciplinary collaboration through developing new graduate and undergraduate courses and involving students in the research activities.