Nonparametric Variable Selection in Smoothing Spline ANOVA Models

研究了非参数平滑和回归模型中的变量选择问题。特别地，发展了一类新的正则化方法，用于平滑样条单因素方差模型中的变量选择和模型拟合。一种这样的方法是“COSSO”，它将一种新的软阈值类型的运算应用于再生核希尔伯特空间中的功能分量。在高斯回归中，COSSO选择正确的模型结构，在某些温和的条件下，概率趋于1。为了处理具有不同类型响应的复杂异质数据集，研究者进一步将新方法扩展到更复杂的统计模型，如广义回归模型、支持向量机和比例风险回归模型。研究了估计量的理论性质，如模型相合性和收敛速度。开发了高效的数值算法和用户友好的软件供公众使用，可变的选择有助于降低建模的维度，提高模型的精度，并更好地理解数据生成的潜在机理。本研究的动机是缺乏非参数变量选择的理论工作和现有方法的局限性。建立了光滑样条型ANOVA模型中同时变量选择和模型估计的统一框架，为相关变分方法提供了新的理论。这项工作拓宽了对非参数平滑方法的传统理解，并最终将有助于产生统计推断中的新方法。在实践中，医学和生物学等现代科学产生的高维大数据集，往往具有数十或数百个变量，需要更复杂的工具来进行降维和模型估计。在这项工作中开发的方法论已经在一些实际问题中得到了成功的应用，它可能会在各个领域产生重大影响。

The investigator studies the variable selection problem in nonparametric smoothing and regression models. In particular, a class of new regularization methods is developed for simultaneous variable selection and model fitting in the smoothing spline ANOVA models. One such method is the "cosso", which applies a novel soft thresholding type operation to the functional components in a reproducing kernel Hilbert space. In Gaussian regression, the cosso selects the correct model structure with the probability tending to one under certain mild conditions. To handle complex heterogeneous datasets with various types of responses, the investigator further extends the new methods to more complicated statistical models, such as generalized regression models, support vector machines, and proportional hazards regression models. Theoretical properties of the estimators like model consistency and the rate of convergence are investigated. Efficient numerical algorithms and user-friendly software are developed for public use.Variable selection helps to reduce the dimension of model building, to improve the model accuracy, and to better understand the underlying mechanism that generates data. This research is motivated by the lack oftheoretical work in nonparametric variable selection and the limits of existing approaches. The investigator establishes a unified framework for simultaneous variable selection and model estimation in smoothing spline ANOVA models, and contributes new theories to related variational methods. This work broadens the traditional understanding of nonparametric smoothing approaches, and eventually will help to generate new methods in statistical inference. In practice, high dimensional large datasets produced in modern sciences such as in medicine and biology, often with tens or hundreds of variables, demand more sophisticated tools for dimension reduction and model estimation. The methodology developed in this work already has successful applications in some real problems, and it will potentially make a significant impact in various fields.