Nonparametric Variable Selection in Smoothing Spline ANOVA Models

研究了非参数平滑和回归模型中的变量选择问题。特别地，提出了一类新的正则化方法，用于光滑样条方差分析模型中同时进行变量选择和模型拟合。一种这样的方法是“cosso”，其将新颖的软阈值类型操作应用于再生核希尔伯特空间中的功能分量。在高斯回归中，cosso选择正确的模型结构，在一定的温和条件下概率趋于1。为了处理具有各种类型响应的复杂异构数据集，研究人员进一步将新方法扩展到更复杂的统计模型，例如广义回归模型，支持向量机和比例风险回归模型。估计的理论性质，如模型的一致性和收敛速度进行了研究。本署开发了高效率的数值算法和易于使用的软件，供公众使用。变量的选取有助减少建立模型的规模、提高模型的准确性，以及加深对产生数据的基本机制的了解。这项研究的动机是缺乏理论工作的非参数变量选择和现有的方法的局限性。建立了光滑样条方差分析模型中变量选择和模型估计同时进行的统一框架，为相关变分方法提供了新的理论。这项工作拓宽了传统的非参数平滑方法的理解，并最终将有助于产生新的方法在统计推断。在实践中，在医学和生物学等现代科学中产生的高维大型数据集通常具有数十或数百个变量，需要更复杂的降维和模型估计工具。在这项工作中开发的方法已经成功地应用在一些真实的问题，它将在各个领域产生重大影响。

The investigator studies the variable selection problem in nonparametric smoothing and regression models. In particular, a class of new regularization methods is developed for simultaneous variable selection and model fitting in the smoothing spline ANOVA models. One such method is the "cosso", which applies a novel soft thresholding type operation to the functional components in a reproducing kernel Hilbert space. In Gaussian regression, the cosso selects the correct model structure with the probability tending to one under certain mild conditions. To handle complex heterogeneous datasets with various types of responses, the investigator further extends the new methods to more complicated statistical models, such as generalized regression models, support vector machines, and proportional hazards regression models. Theoretical properties of the estimators like model consistency and the rate of convergence are investigated. Efficient numerical algorithms and user-friendly software are developed for public use.Variable selection helps to reduce the dimension of model building, to improve the model accuracy, and to better understand the underlying mechanism that generates data. This research is motivated by the lack oftheoretical work in nonparametric variable selection and the limits of existing approaches. The investigator establishes a unified framework for simultaneous variable selection and model estimation in smoothing spline ANOVA models, and contributes new theories to related variational methods. This work broadens the traditional understanding of nonparametric smoothing approaches, and eventually will help to generate new methods in statistical inference. In practice, high dimensional large datasets produced in modern sciences such as in medicine and biology, often with tens or hundreds of variables, demand more sophisticated tools for dimension reduction and model estimation. The methodology developed in this work already has successful applications in some real problems, and it will potentially make a significant impact in various fields.