Approximation Methods for Supervised Learning
Approximation Methods for Supervised Learning
复制标题
DOI：
10.1007/s10208-004-0158-6
复制
发表时间：
2006-02
影响因子：
3
通讯作者：
R. DeVore;G. Kerkyacharian;D. Picard;V. Temlyakov
中科院分区：
文献类型：
--
作者：
R. DeVore;G. Kerkyacharian;D. Picard;V. Temlyakov
设ρ是定义在空间Z：= X × Y上的未知Borel测度，其中X为零，Y = [-M，M].给定根据ρ绘制的m个样本的集合z =（xi，yi），考虑使用这些样本估计回归函数fρ的问题。主要的重点是了解什么是近似率，无论是在预期或概率测量， 可以在给定的先验fρ∈ Θ下获得，即，在假设fρ在集合Θ中的情况下，以及获得最优或半最优（最多为10 ms）结果的可能算法。 以m表示的最优衰减率是针对许多先验建立的，这些先验要么以fρ的光滑度表示，要么以几种方式之一测量其近似率。 这个最佳速率由两种类型的结果决定。 上界是使用各种近似工具建立的，如熵，宽度，线性和非线性近似。 下界证明使用Kullback-Leibler信息与Fano不等式和某种类型的熵。 区分算法，采用知识的建设中的估计和那些不。第二类算法是普遍最优的一定范围内的先验。
Let ρ be an unknown Borel measure defined on the space Z := X × Y with X ⊂ ℝdand Y = [-M,M]. Given a set z of m samples zi=(xi,yi) drawn according to ρ, the problem of estimating a regression function fρusing these samples is considered. The main focus is to understand what is the rate of approximation, measured either in expectation or probability, that can be obtained under a given prior fρ∈ Θ, i.e., under the assumption that fρis in the set Θ, and what are possible algorithms for obtaining optimal or semioptimal (up to logarithms) results. The optimal rate of decay in terms of m is established for many priors given either in terms of smoothness of fρor its rate of approximation measured in one of several ways. This optimal rate is determined by two types of results. Upper bounds are established using various tools in approximation such as entropy, widths, and linear and nonlinear approximation. Lower bounds are proved using Kullback-Leibler information together with Fano inequalities and a certain type of entropy. A distinction is drawn between algorithms which employ knowledge of the prior in the construction of the estimator and those that do not. Algorithms of the second type which are universally optimal for a certain range of priors are given.