Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inference
Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inference
复制标题
DOI：
10.48550/arxiv.2310.00532
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
Licong Lin;Mufang Ying;Suvrojit Ghosh;K. Khamaru;Cun-Hui Zhang
中科院分区：
文献类型：
--
作者：
Licong Lin;Mufang Ying;Suvrojit Ghosh;K. Khamaru;Cun-Hui Zhang
在自适应地收集数据时，统计中的估计和推断带来了巨大的挑战。即使在线性模型中，普通最小二乘(OLS)估计器对于单坐标估计也可能不表现出渐近正态，并且存在膨胀误差。最近的一个极小极大下界突出了这个问题，它表明，当数据被允许任意自适应时，估计单个坐标的误差可以比当它们是I.D.的情况下扩大$SQRT{d}$的倍数。我们的工作探索了在使用I.I.D.和使用I.I.D.之间在估计性能上的显著差异。和自适应数据。我们研究了数据收集的自适应程度如何影响高维线性模型中估计低维参数分量的性能。我们在数据收集机制上确定了低维参数分量的估计误差与I.I.D.中的对应分量相匹配的条件。设置，最多取决于自适应程度的系数。我们证明了OLS或基于中心数据的OLS可以达到这一匹配误差。此外，我们还通过求解一个两阶段自适应线性估计方程(TALE)，提出了一种新的单坐标推断估计器。在数据采集的自适应性较弱的情况下，我们建立了所提出的估计的渐近正态性质。
Estimation and inference in statistics pose significant challenges when data are collected adaptively. Even in linear models, the Ordinary Least Squares (OLS) estimator may fail to exhibit asymptotic normality for single coordinate estimation and have inflated error. This issue is highlighted by a recent minimax lower bound, which shows that the error of estimating a single coordinate can be enlarged by a multiple of $\sqrt{d}$ when data are allowed to be arbitrarily adaptive, compared with the case when they are i.i.d. Our work explores this striking difference in estimation performance between utilizing i.i.d. and adaptive data. We investigate how the degree of adaptivity in data collection impacts the performance of estimating a low-dimensional parameter component in high-dimensional linear models. We identify conditions on the data collection mechanism under which the estimation error for a low-dimensional parameter component matches its counterpart in the i.i.d. setting, up to a factor that depends on the degree of adaptivity. We show that OLS or OLS on centered data can achieve this matching error. In addition, we propose a novel estimator for single coordinate inference via solving a Two-stage Adaptive Linear Estimating equation (TALE). Under a weaker form of adaptivity in data collection, we establish an asymptotic normality property of the proposed estimator.