Extending the Use of MDL for High-Dimensional Problems: Variable Selection, Robust Fitting, and Additive Modeling
Extending the Use of MDL for High-Dimensional Problems: Variable Selection, Robust Fitting, and Additive Modeling
复制标题
扩展 MDL 在高维问题中的使用：变量选择、鲁棒拟合和加性建模
DOI：
10.1109/icassp43922.2022.9746206
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Thomas C.M. Lee
中科院分区：
文献类型：
--
作者：
Zhenyu Wei;Raymond K. W. Wong;Thomas C.M. Lee
在信号处理和统计文献中，最小描述长度（MDL）原则是选择模型复杂性的流行工具。成功的例子包括线性回归中的信号去噪和变量选择，相应的 MDL 解决方案通常具有一致的特性并产生非常有希望的经验结果。本文证明 MDL 可以自然地扩展到高维设置，其中预测变量的数量 p 大于观测值的数量 n。它首先考虑线性回归的情况，然后考虑数据中的异常值，最后扩展到非参数加性模型的稳健拟合。数值实验的结果证明了 MDL 方法的效率和有效性。
In the signal processing and statistics literature, the minimum description length (MDL) principle is a popular tool for choosing model complexity. Successful examples include signal denoising and variable selection in linear regression, for which the corresponding MDL solutions often enjoy consistent properties and produce very promising empirical results. This paper demonstrates that MDL can be extended naturally to the high-dimensional setting, where the number of predictors p is larger than the number of observations n. It first considers the case of linear regression, then allows for outliers in the data, and lastly extends to the robust fitting of nonparametric additive models. Results from numerical experiments are presented to demonstrate the efficiency and effectiveness of the MDL approach.