Concentration of Maxima and Fundamental Limits in High-Dimensional Testing and Inference
Concentration of Maxima and Fundamental Limits in High-Dimensional Testing and Inference
复制标题
高维测试和推理中最大值和基本极限的集中
DOI：
10.1007/978-3-030-80964-5
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Stoev, Stilian
中科院分区：
文献类型：
--
作者：
Gao, Zheng;Stoev, Stilian
本文介绍了稀疏信号问题中相变现象的一些新结果和最新进展。主要的主题是在高维测试和推理的基本限制的研究。自Ingster（1998）和Donoho and Jin（2004）的开创性著作以来，该主题在文献中受到了大量关注，Ji and Jin（2012）; Genovese et al.（2012）; Jin et al.（2014）; Arias-Castro and Chen（2017）; Butucea et al.（2018年）。这些工作，除其他外，已经发现了一些基本的限制，在所谓的大海捞针问题，其中一个稀疏的信号与高维加性噪声观察。在这种情况下，出现了两个典型的问题-信号检测和信号支持恢复。信号检测是指一个全局假设检验问题，其相当于确定在其任何维度上存在非零信号。另一方面，支持恢复可以被看作是一个多测试问题，其中针对每个感兴趣的信号位置测试非零信号的存在，或者可替代地，作为一个推断问题，其目的是估计信号支持，即非零信号分量的位置。这些问题的基本限制研究在所谓的高维渐近制度，其中维度p的基本信号增长到无穷大，和样本量n是有界的或增长缓慢相对于p.从概率的角度来看，这些上述基本限制被称为渐近0 - 1型法律，作为维数发散。也就是说，考虑一个稀疏信号，对于某个参数β∈（0，1），支持大小为p1− β。对于某些r> 0和适当的单调非递减函数（·），通过（pr）来参数化非零信号幅度。然后，对于广泛的误差分布和统计问题，人们遇到了一个急剧的过渡之间的制度，其中问题是可解的和不可解的信号幅度r和信号稀疏度β的依赖。更准确地说，存在一个边界函数f（β），如果信号幅度在边界之上，r> f（β），那么这个问题可以用合适的统计过程在p→∞时消失损失的情况下解决。另一方面，如果信号低于同一边界，即r< f（β），则所有统计程序都不能提供具有消失损失的解，因为p→∞。当然，取决于是否考虑检测（测试）或支持恢复（推断）问题，
This text presents a collection of new results and recent developments on the phasetransition phenomena in sparse signal problems. The main theme is the study of the fundamental limits in high-dimensional testing and inference. Since the seminal works of Ingster (1998) and Donoho and Jin (2004), the subject has received a lot of attention in the literature with important contributions from Ji and Jin (2012); Genovese et al.(2012); Jin et al.(2014); Arias-Castro and Chen (2017); Butucea et al.(2018). These works, among many others, have discovered some fundamental limits in the so-called needle in haystack problems, where a sparse signal is observed with high-dimensional additive noise. In this setting, two archetypal problems arise—the signal detection and signal support recovery. The signal detection refers to a global hypothesis testing problem that amounts to determining the presence of non-zero signal in any of its dimensions. The support recovery, on the other hand, can be seen either as a multiple testing problem where the presence of non-zero signal is tested for each signal location of interest, or alternatively, as an inference problem that aims to estimate the signal support, ie, the locations of the non-zero signal components. The fundamental limits of these problems are studied in the so-called high-dimensional asymptotic regime where the dimension p of the underlying signal grows to infinity, and the sample size n is either bounded or grows slowly relative to p. From a probabilistic perspective, these aforementioned fundamental limits are stated as asymptotic zero-one type laws, as dimensionality diverges. Namely, consider a sparse signal with support size on the order of p1− β for some parameter β∈(0, 1). Parameterize the non-zero signal amplitude by (pr), for some r> 0 and a suitable monotone non-decreasing function (·). Then, for a broad range of error distributions and statistical problems, one encounters a sharp transition between the regimes where the problem is solvable and unsolvable depending on the signal magnitude r and signal sparsity β. More precisely, there exists a boundary function f (β) such that if the signal magnitudes are above the boundary, r> f (β), then the problem can be solved with vanishing loss as p→∞, with a suitable statistical procedure. On the other hand, if the signal is below that same boundary, ie, r< f (β), all statistical procedures fail to provide a solution with a vanishing loss, as p→∞. Of course, depending on whether one considers the detection (testing) or support recovery (inference) problems, different loss functions vii