Component selection and smoothing in multivariate nonparametric regression
Component selection and smoothing in multivariate nonparametric regression
复制标题
DOI：
10.1214/009053606000000722
复制
发表时间：
2006-10-01
影响因子：
4.5
通讯作者：
Zhang, Hao Helen
中科院分区：
文献类型：
--
作者：
Lin, Yi;Zhang, Hao Helen
本文在光滑样条方差分析的框架下，提出了一种新的多元非参数回归模型的模型选择和模型拟合方法。“COSSO”是一种正则化方法，其惩罚函数是分量范数之和，而不是传统光滑样条方法中采用的平方范数。COSSO为线性模型和平滑样条方差分析模型中的模型选择提供了一个统一的框架。理论性质，如COSSO估计的存在性和收敛速度，进行了研究。在张量积设计与周期函数的特殊情况下，详细的分析表明，COSSO通过应用一种新的软阈值型操作的功能组件的模型选择。我们给出了一个等价的制定COSSO估计自然导致一个迭代算法。我们比较COSSO与MARS，一种流行的方法，建立功能方差分析模型，在模拟和真实的例子。COSSO方法可以扩展到分类问题，我们比较其性能与机器学习算法的数量在真实的数据集。COSSO在这些研究中表现出非常有竞争力的性能。
We propose a new method for model selection and model fitting in multivariate nonparametric regression models, in the framework of smoothing spline ANOVA. The "COSSO" is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in the traditional smoothing spline method. The COSSO provides a unified framework for several recent proposals for model selection in linear models and smoothing spline ANOVA models. Theoretical properties, such as the existence and the rate of convergence of the COSSO estimator, are studied. In the special case of a tensor product design with periodic functions, a detailed analysis reveals that the COSSO does model selection by applying a novel soft thresholding type operation to the function components. We give an equivalent formulation of the COSSO estimator which leads naturally to an iterative algorithm. We compare the COSSO with MARS, a popular method that builds functional ANOVA models, in simulations and real examples. The COSSO method can be extended to classification problems and we compare its performance with those of a number of machine learning algorithms on real datasets. The COSSO gives very competitive performance in these studies.