On the Estimation of Derivatives Using Plug-in Kernel Ridge Regression Estimators
On the Estimation of Derivatives Using Plug-in Kernel Ridge Regression Estimators
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Zejian Liu;Meng Li
中科院分区：
文献类型：
--
作者：
Zejian Liu;Meng Li
研究了回归函数导数的估计问题，回归函数作为未知函数的关键非参数泛函有着广泛的应用。标准分析可以针对特定的导数阶数进行定制，并且参数调整仍然是一个艰巨的挑战，特别是对于高阶导数。在这篇文章中，我们提出了一个简单的插件核岭回归（KRR）估计在非参数回归随机设计，广泛适用于多维支持和任意混合偏导数。我们提供了一个非渐近分析，以研究的行为，建议的估计，在一个统一的方式，包括回归函数及其衍生物，导致两个误差界下的强$L_\infty$范数的一般类的内核。在一个具体的例子中，专门为内核与多项式衰减的特征值，建议的估计恢复极小极大最优速率的对数因子估计的衍生物的功能在H\“older和Sobolev类。有趣的是，所提出的估计器实现了最佳的收敛速度与相同的选择的调谐参数的任何阶导数。因此，建议的估计享有\textit{插件属性}的衍生工具，因为它自动适应的阶导数估计，使易于调整在实践中。我们的模拟研究表明，相对于现有的几种方法，该方法具有良好的有限样本性能，并证实了其极大极小最优性的理论研究结果。
We study the problem of estimating the derivatives of a regression function, which has a wide range of applications as a key nonparametric functional of unknown functions. Standard analysis may be tailored to specific derivative orders, and parameter tuning remains a daunting challenge particularly for high-order derivatives. In this article, we propose a simple plug-in kernel ridge regression (KRR) estimator in nonparametric regression with random design that is broadly applicable for multi-dimensional support and arbitrary mixed-partial derivatives. We provide a non-asymptotic analysis to study the behavior of the proposed estimator in a unified manner that encompasses the regression function and its derivatives, leading to two error bounds for a general class of kernels under the strong $L_\infty$ norm. In a concrete example specialized to kernels with polynomially decaying eigenvalues, the proposed estimator recovers the minimax optimal rate up to a logarithmic factor for estimating derivatives of functions in H\"older and Sobolev classes. Interestingly, the proposed estimator achieves the optimal rate of convergence with the same choice of tuning parameter for any order of derivatives. Hence, the proposed estimator enjoys a \textit{plug-in property} for derivatives in that it automatically adapts to the order of derivatives to be estimated, enabling easy tuning in practice. Our simulation studies show favorable finite sample performance of the proposed method relative to several existing methods and corroborate the theoretical findings on its minimax optimality.