Rare Feature Selection in High Dimensions
Rare Feature Selection in High Dimensions
复制标题
DOI：
10.1080/01621459.2020.1796677
复制
发表时间：
2018-03
影响因子：
3.7
通讯作者：
Xiaohan Yan;J. Bien
中科院分区：
文献类型：
--
作者：
Xiaohan Yan;J. Bien
摘要在现代预测问题中是许多预测变量是很少发生的事件的计数。潜水区域，从自然语言处理（例如，稀有词）到生物学（例如，稀有物种）。对特征的稀有性进行核算，可以大大降低分析的有效性。该编码具有相似之处，我们将我们的方法应用于TripAdvisor的数据，在其中，我们根据相关评论的文本预测酒店的数值等级。相比之下，通过有效使用稀有单词，如果它们是稀有的，则无法识别出高度预测的词本文可在线获得。
Abstract It is common in modern prediction problems for many predictor variables to be counts of rarely occurring events. This leads to design matrices in which many columns are highly sparse. The challenge posed by such “rare features” has received little attention despite its prevalence in diverse areas, ranging from natural language processing (e.g., rare words) to biology (e.g., rare species). We show, both theoretically and empirically, that not explicitly accounting for the rareness of features can greatly reduce the effectiveness of an analysis. We next propose a framework for aggregating rare features into denser features in a flexible manner that creates better predictors of the response. Our strategy leverages side information in the form of a tree that encodes feature similarity. We apply our method to data from TripAdvisor, in which we predict the numerical rating of a hotel based on the text of the associated review. Our method achieves high accuracy by making effective use of rare words; by contrast, the lasso is unable to identify highly predictive words if they are too rare. A companion R package, called rare, implements our new estimator, using the alternating direction method of multipliers. Supplementary materials for this article are available online.