Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy–Krause variation
Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy–Krause variation
复制标题
DOI：
10.1214/20-aos1977
复制
发表时间：
2019-03
期刊：
影响因子：
--
通讯作者：
Billy Fang;Adityanand Guntuboyina;B. Sen
中科院分区：
文献类型：
--
作者：
Billy Fang;Adityanand Guntuboyina;B. Sen
当协变量是$d$维（$d\geq1$）时，我们考虑非参数回归问题。在本文中，我们在这种设定下引入并研究了两种非参数最小二乘估计量（LSE）——完全单调最小二乘估计量和约束哈代 - 克劳斯变差最小二乘估计量。我们表明这两种最小二乘估计量分别是一元保序回归和一元全变差去噪到多维的自然推广。我们讨论了从$n$个数据点得到的这两种最小二乘估计量的特征和计算。我们在平方误差损失和固定均匀格点设计下对它们的风险性质进行了详细研究。我们表明，这些最小二乘估计量的有限样本风险总是由依赖于$d$的对数因子修正后的$n^{-2/3}$从上方界定；因此这些非参数最小二乘估计量在一定程度上避免了维数灾难。我们还证明了几乎匹配的极小极大下界。此外，我们说明这些最小二乘估计量在拟合矩形分段常数函数时特别有用。具体而言，我们表明当真实函数可以由一个具有不太多常数段的矩形分段常数完全单调函数很好地逼近时，完全单调最小二乘估计量的风险几乎是参数性的（至多为$1/n$，直到对数因子）。对于矩形分段常数函数的一个简单子类，约束哈代 - 克劳斯变差最小二乘估计量也被证明有类似的结果成立。我们相信所提出的最小二乘估计量提供了一种使用凸优化估计多元函数的新方法，该方法在一定程度上避免了维数灾难。
We consider the problem of nonparametric regression when the covariate is $d$-dimensional, where $d \geq 1$. In this paper we introduce and study two nonparametric least squares estimators (LSEs) in this setting---the entirely monotonic LSE and the constrained Hardy-Krause variation LSE. We show that these two LSEs are natural generalizations of univariate isotonic regression and univariate total variation denoising, respectively, to multiple dimensions. We discuss the characterization and computation of these two LSEs obtained from $n$ data points. We provide a detailed study of their risk properties under the squared error loss and fixed uniform lattice design. We show that the finite sample risk of these LSEs is always bounded from above by $n^{-2/3}$ modulo logarithmic factors depending on $d$; thus these nonparametric LSEs avoid the curse of dimensionality to some extent. We also prove nearly matching minimax lower bounds. Further, we illustrate that these LSEs are particularly useful in fitting rectangular piecewise constant functions. Specifically, we show that the risk of the entirely monotonic LSE is almost parametric (at most $1/n$ up to logarithmic factors) when the true function is well-approximable by a rectangular piecewise constant entirely monotone function with not too many constant pieces. A similar result is also shown to hold for the constrained Hardy-Krause variation LSE for a simple subclass of rectangular piecewise constant functions. We believe that the proposed LSEs yield a novel approach to estimating multivariate functions using convex optimization that avoid the curse of dimensionality to some extent.