A high-dimensional Wilks phenomenon
A high-dimensional Wilks phenomenon
复制标题
高维威尔克斯现象
DOI：
--
复制
发表时间：
2011
期刊：
影响因子：
--
通讯作者：
P. Massart
中科院分区：
文献类型：
--
作者：
S. Boucheron;P. Massart
Wilks的一个定理断言，在光滑参数密度估计中，抽样分布的最大似然和似然之间的差收敛于卡方分布，其中自由度数与模型维度重合。这一观察结果是一些拟合优度测试程序和一些经典模型选择方法的核心。本文描述了有界对比度优化过程中Wilks现象的一个非渐近形式。利用独立随机变量一般函数的浓度不等式，证明了在有界对比度最小化(如在统计学习理论中)，模型中真实风险的最小化的经验风险与经验风险的最小值(超额经验风险)之间的差值满足类Bernstein不等式，其中方差项反映模型的维度，尺度项反映噪声条件。从数理统计的角度来看，这一结果的意义来自于最近的一项观察，即当通过惩罚进行模型选择时，如果要提供关于预测误差的非渐近保证，则额外的经验风险代表着最小的惩罚。从经验过程理论的角度，描述了有界无中心(实际上非正)经验过程的上确界的一个集中不等式。结合目前对M估计的经典分析(建立在经验过程上界的TALAGRAND不等式上)和独立随机变量函数的通用矩不等式，本文发展了一个传统工具所不能企及的真正的Bernstein型不等式。
A theorem by Wilks asserts that in smooth parametric density estimation the difference between the maximum likelihood and the likelihood of the sampling distribution converges toward a Chi-square distribution where the number of degrees of freedom coincides with the model dimension. This observation is at the core of some goodness-of-fit testing procedures and of some classical model selection methods. This paper describes a non-asymptotic version of the Wilks phenomenon in bounded contrast optimization procedures. Using concentration inequalities for general functions of independent random variables, it proves that in bounded contrast minimization (as for example in Statistical Learning Theory), the difference between the empirical risk of the minimizer of the true risk in the model and the minimum of the empirical risk (the excess empirical risk) satisfies a Bernstein-like inequality where the variance term reflects the dimension of the model and the scale term reflects the noise conditions. From a mathematical statistics viewpoint, the significance of this result comes from the recent observation that when using model selection via penalization, the excess empirical risk represents a minimum penalty if non-asymptotic guarantees concerning prediction error are to be provided. From the perspective of empirical process theory, this paper describes a concentration inequality for the supremum of a bounded non-centered (actually non-positive) empirical process. Combining the now classical analysis of M-estimation (building on Talagrand’s inequality for suprema of empirical processes) and versatile moment inequalities for functions of independent random variables, this paper develops a genuine Bernstein-like inequality that seems beyond the reach of traditional tools.