Robust Estimation and Model Ensemble Selection

鲁棒估计和模型集成选择

• 批准号: RGPIN-2019-04201
• 负责人: Zamar, Ruben
• 金额: $ 1.82万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=765727
• 关键词: Robust; Estimation; Model; Ensemble; Selection

Consider a data table with n rows - the cases - and d columns - the variables. The classical robustness model assumes that the majority of the cases  are contamination free. Hence, a minority of contaminated cases may need to be identified and filtered. Unfortunately this paradigm is not realistic and collapses in high dimensions. If there is a small and independent probability p that a cell (individual entry in the data table) is contaminated then the probability that a case (a row in the data table) is contaminated is 1-(1-p)^d which can quickly go over 0.5. For example, if p=0.01 and d=100 the probability is 0.63397.  The PhD thesis of my former student Fatemah Alqallaf, brings attention to this problem called propagation of outliers. Alqallaf, Van Aelst, Yohai and Zamar (2009) continue this research and shows that traditional high breakdown point estimates are no longer resistant against independent contamination. Hence, the PhD thesis of my former student Andy Leung constructed multivariate location and scatter estimates which are resistant to casewise and cellwise outliers. Extensions of this approach to other multivariate models including cluster analysis will be considered by my new graduate students Glenn McGuinness and Malvika Mitra. Cheap collection/storage of data produced  variable-rich-case-poor data. It is common to find that a large number of the variables in these datasets are noise variables which hurt rather than help inference tasks. Moreover, the remaining useful variables may be partially redundant and subsets of these variables may predict better than the full set. Then it may be better to ensemble several models based on diverse subsets of variables. This lead to  a very broad research endeavor - ensemble selection - a generalization of model selection. The PhD thesis of my former student Jabed Tomal proposes an ad-hoc  procedure called phalanxes. My PhD student Anthony Christidis will consider a more structured selection of optimal ensembles by optimizing a new loss function that penalizes lack of sparseness and  lack of diversity of the models selected for the ensemble. With the help of several grad students I wish to study extensions to linear and nonlinear regression and classification. I am also interested in  cluster analysis. A popular clustering method is K-means. However, this method is not resistant against outliers.  My PhD student Juan D. Gonzalez (University of Buenos Aires, Argentina) is developing a robust alternative by using the Tau-scale (Yohai and Zamar, 1989) instead of the average of the distances from the points to their cluster centers. Application of this procedure to the processing of digitized images is also part of his research project.

考虑一个有n行（大小写）和d列（变量）的数据表。经典的鲁棒性模型假设大多数情况下是无污染的。因此，可能需要识别和过滤少数受污染病例。不幸的是，这种范式是不现实的，并在高维中崩溃。如果一个单元格（数据表中的单个条目）被污染的概率很小且独立，那么一个case（数据表中的一行）被污染的概率是1-(1-p)^d，可以很快超过0.5。例如，如果p=0.01, d=100，则概率为0.63397。我以前的学生Fatemah Alqallaf的博士论文，引起了人们对异常值传播问题的关注。Alqallaf、Van Aelst、Yohai和Zamar（2009）继续了这项研究，并表明传统的高击穿点估计不再抵抗独立污染。因此，我以前的学生Andy Leung的博士论文构建了多元位置和散点估计，这些估计可以抵抗个案和单元异常值。我的新研究生Glenn McGuinness和Malvika Mitra将考虑将这种方法扩展到其他多变量模型，包括聚类分析。廉价的数据收集/存储产生了变量丰富的数据。通常会发现，这些数据集中的大量变量是噪声变量，不利于而不是有助于推理任务。此外，剩余的有用变量可能是部分冗余的，这些变量的子集可能比完整的集合预测得更好。那么基于不同的变量子集集成几个模型可能会更好。这导致了一个非常广泛的研究努力-集成选择-模型选择的概括。我以前的学生贾贝德·托马尔的博士论文提出了一种特殊的程序，叫做方阵。我的博士生Anthony Christidis将通过优化一个新的损失函数来考虑更结构化的最优集成选择，该损失函数会惩罚为集成选择的模型缺乏稀疏性和缺乏多样性。在几位研究生的帮助下，我希望学习线性和非线性回归与分类的扩展。我对聚类分析也很感兴趣。一种流行的聚类方法是K-means。然而，这种方法不能抵抗异常值。我的博士生Juan D. Gonzalez（阿根廷布宜诺斯艾利斯大学）正在开发一种强大的替代方法，通过使用tau尺度（Yohai和Zamar， 1989）来代替从点到集群中心的平均距离。将此程序应用于数字化图像的处理也是其研究项目的一部分。