ML-1M++: MovieLens-Compatible Additional Preferences for More Robust Offline Evaluation of Sequential Recommenders
ML-1M++: MovieLens-Compatible Additional Preferences for More Robust Offline Evaluation of Sequential Recommenders
复制标题
DOI：
10.1145/3511808.3557643
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Kazutoshi Umemoto
中科院分区：
文献类型：
--
作者：
Kazutoshi Umemoto
顺序推荐的任务是根据目标用户过去的交互顺序，预测他/她的下一个交互项目。通常，顺序推荐器是离线评估的，每个序列中的最后一项作为对应用户测试示例的唯一正确（相关）标签。然而，人们对偏好数据的稀疏性如何影响离线评估结果的稳健性知之甚少。为了帮助研究人员解决这个问题，我们通过众包收集了额外的偏好数据。具体来说，我们提出了一个针对顺序推荐任务的评估接口，并要求人群工作人员评估MovieLens 1M（一个常用的数据集）中每个候选项目的（潜在）相关性。为了建立一个更健壮的评估方法，我们发布了收集到的偏好数据，我们称之为ML-1M++，以及评估接口的代码。
Sequential recommendation is the task of predicting the next interacted item of a target user, given his/her past interaction sequence. Conventionally, sequential recommenders are evaluated offline with the last item in each sequence as the sole correct (relevant) label for the testing example of the corresponding user. However, little is known about how this sparsity of preference data affects the robustness of the offline evaluation's outcomes. To help researchers address this, we collect additional preference data via crowdsourcing. Specifically, we propose an assessment interface tailored to the sequential recommendation task and ask crowd workers to assess the (potential) relevance of each candidate item in MovieLens 1M, a commonly used dataset. Toward establishing a more robust evaluation methodology, we release the collected preference data, which we call ML-1M++, as well as the code of the assessment interface.