Correcting the bias in least squares regression with volume-rescaled sampling
Correcting the bias in least squares regression with volume-rescaled sampling
复制标题
DOI：
--
复制
发表时间：
2018-10
期刊：
影响因子：
--
通讯作者：
Michal Derezinski;Manfred K. Warmuth;Daniel J. Hsu
中科院分区：
文献类型：
--
作者：
Michal Derezinski;Manfred K. Warmuth;Daniel J. Hsu
考虑线性回归，其中样本是由\(R^d\times R\)上的未知分布生成的。在对噪声没有任何假设的情况下，对于任何独立同分布（i.i.d.）样本，线性最小二乘解相对于整个分布上的最小二乘最优解通常是有偏的。然而，我们表明，如果任何大小为\(k\)的独立同分布样本通过一个特定的小附加样本进行扩充，那么合并样本的解就会变得无偏。当附加样本由根据输入分布联合抽取的\(d\)个点组成，且该输入分布由这些点所张成的平方体积进行了重新缩放时，我们证明了这一点。此外，当数据分布仅通过独立同分布样本已知时，我们提出了从这种体积重新缩放分布中采样的算法。
Consider linear regression where the examples are generated by an unknown distribution on $R^d\times R$. Without any assumptions on the noise, the linear least squares solution for any i.i.d. sample will typically be biased w.r.t. the least squares optimum over the entire distribution. However, we show that if an i.i.d. sample of any size k is augmented by a certain small additional sample, then the solution of the combined sample becomes unbiased. We show this when the additional sample consists of d points drawn jointly according to the input distribution that is rescaled by the squared volume spanned by the points. Furthermore, we propose algorithms to sample from this volume-rescaled distribution when the data distribution is only known through an i.i.d sample.