On the Generalizability and Predictability of Recommender Systems
On the Generalizability and Predictability of Recommender Systems
复制标题
DOI：
10.48550/arxiv.2206.11886
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Duncan C. McElfresh;Sujay Khandagale;Jonathan Valverde;John P. Dickerson;Colin White
中科院分区：
文献类型：
--
作者：
Duncan C. McElfresh;Sujay Khandagale;Jonathan Valverde;John P. Dickerson;Colin White
虽然机器学习的其他领域已经实现了越来越多的自动化，但设计一个高性能的推荐系统仍然需要大量的人力。此外，最近的工作表明，现代推荐系统算法并不总是改善良好的调整基线。一个自然的后续问题是，“我们如何为新的数据集和性能指标选择正确的算法？“在这项工作中，我们首先对推荐系统方法进行了首次大规模研究，通过比较85个数据集和315个指标的18种算法和100组超参数。我们发现，最好的算法和超参数高度依赖于数据集和性能指标，然而，每个算法的性能和数据集的各种元特征之间也有很强的相关性。受这些发现的启发，我们创建了RecZilla，这是一种推荐系统的元学习方法，它使用一个模型来预测新的、看不见的数据集的最佳算法和超参数。通过使用比以前更多的元训练数据，RecZilla能够在面对新的推荐系统应用程序时大大降低人类参与的程度。我们不仅发布了我们的代码和预训练的RecZilla模型，还发布了我们所有的原始实验结果，以便从业者可以根据他们想要的性能指标训练RecZilla模型。https://github.com/naszilla/reczilla
While other areas of machine learning have seen more and more automation, designing a high-performing recommender system still requires a high level of human effort. Furthermore, recent work has shown that modern recommender system algorithms do not always improve over well-tuned baselines. A natural follow-up question is,"how do we choose the right algorithm for a new dataset and performance metric?"In this work, we start by giving the first large-scale study of recommender system approaches by comparing 18 algorithms and 100 sets of hyperparameters across 85 datasets and 315 metrics. We find that the best algorithms and hyperparameters are highly dependent on the dataset and performance metric, however, there are also strong correlations between the performance of each algorithm and various meta-features of the datasets. Motivated by these findings, we create RecZilla, a meta-learning approach to recommender systems that uses a model to predict the best algorithm and hyperparameters for new, unseen datasets. By using far more meta-training data than prior work, RecZilla is able to substantially reduce the level of human involvement when faced with a new recommender system application. We not only release our code and pretrained RecZilla models, but also all of our raw experimental results, so that practitioners can train a RecZilla model for their desired performance metric: https://github.com/naszilla/reczilla.