Regression in tensor product spaces by the method of sieves
Regression in tensor product spaces by the method of sieves
复制标题
DOI：
10.1214/23-ejs2188
复制
发表时间：
2022-06
影响因子：
1.1
通讯作者：
Tianyu Zhang;N. Simon
中科院分区：
文献类型：
--
作者：
Tianyu Zhang;N. Simon
条件平均值的估计（将一组特征与感兴趣的结果联系起来）是一项基本的统计任务。虽然人们对灵活的非参数过程很感兴趣，但在许多经典的非参数函数空间（例如多元索博列夫空间）中进行有效估计可能非常困难——无论是在统计上还是在计算上——尤其是当特征数量很大时。在本文中，我们提出了用于非参数张量积空间中回归的（惩罚）筛估计量：这些空间更适合多元回归，并允许我们部分避免维数灾难。我们的估计器可以轻松应用于多元非参数问题，并具有吸引人的统计和计算特性。此外，它们可以有效地利用附加结构，例如特征稀疏性。在这份手稿中，我们给出了理论保证，表明我们的估计器的预测性能在维度上具有良好的扩展性。此外，我们还提供了数值示例，以将所提出的估计器的有限样本性能与几种流行的机器学习方法进行比较。
Estimation of a conditional mean (linking a set of features to an outcome of interest) is a fundamental statistical task. While there is an appeal to flexible nonparametric procedures, effective estimation in many classical nonparametric function spaces (e.g., multivariate Sobolev spaces) can be prohibitively difficult -- both statistically and computationally -- especially when the number of features is large. In this paper, we present (penalized) sieve estimators for regression in nonparametric tensor product spaces: These spaces are more amenable to multivariate regression, and allow us to, in-part, avoid the curse of dimensionality. Our estimators can be easily applied to multivariate nonparametric problems and have appealing statistical and computational properties. Moreover, they can effectively leverage additional structures such as feature sparsity. In this manuscript, we give theoretical guarantees, indicating that the predictive performance of our estimators scale favorably in dimension. In addition, we also present numerical examples to compare the finite-sample performance of the proposed estimators with several popular machine learning methods.