Multiobjective optimization for model selection in kernel methods in regression.
Multiobjective optimization for model selection in kernel methods in regression.
复制标题
DOI：
10.1109/tnnls.2013.2297686
复制
发表时间：
2014-10
影响因子：
10.4
通讯作者：
Martinez AM
中科院分区：
文献类型：
--
作者：
You D;Benitez-Quiroz CF;Martinez AM
回归在许多科学和工程问题中发挥着重要作用。回归的目标是从一组具有已知结果的样本向量中学习未知的基础函数。近年来，回归中的核方法促进了非线性函数的估计。然而，两个主要（相互关联的）问题仍然存在。第一个问题是由偏差与方差的权衡给出的。如果用于估计基础函数的模型过于灵活（即模型复杂度高），则方差将非常大。如果模型是固定的（即低复杂度），偏差就会很大。第二个问题是定义一种选择适当的核函数参数的方法。为了解决这两个问题，本文推导了一种新的平滑核准则，该准则测量估计函数的粗糙度作为模型复杂性的度量。然后，我们使用多目标优化来导出选择该内核参数的标准。该标准的目标是找到学习函数的偏差和方差之间的权衡。也就是说，目标是在控制模型复杂性的同时增加模型拟合度。我们利用机器学习、模式识别和计算机视觉中的各种问题提供广泛的实验评估。结果表明，与现有技术的方法相比，所提出的方法产生更小的估计误差。
Regression plays a major role in many scientific and engineering problems. The goal of regression is to learn the unknown underlying function from a set of sample vectors with known outcomes. In recent years, kernel methods in regression have facilitated the estimation of nonlinear functions. However, two major (interconnected) problems remain open. The first problem is given by the bias-vs-variance trade-off. If the model used to estimate the underlying function is too flexible (i.e., high model complexity), the variance will be very large. If the model is fixed (i.e., low complexity), the bias will be large. The second problem is to define an approach for selecting the appropriate parameters of the kernel function. To address these two problems, this paper derives a new smoothing kernel criterion, which measures the roughness of the estimated function as a measure of model complexity. Then, we use multiobjective optimization to derive a criterion for selecting the parameters of that kernel. The goal of this criterion is to find a trade-off between the bias and the variance of the learned function. That is, the goal is to increase the model fit while keeping the model complexity in check. We provide extensive experimental evaluations using a variety of problems in machine learning, pattern recognition and computer vision. The results demonstrate that the proposed approach yields smaller estimation errors as compared to methods in the state of the art.