NASRec: Weight Sharing Neural Architecture Search for Recommender Systems
NASRec: Weight Sharing Neural Architecture Search for Recommender Systems
复制标题
NASRec：推荐系统的权重共享神经架构搜索
DOI：
10.1145/3543507.3583446
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Wen, Wei
中科院分区：
文献类型：
--
作者：
Zhang, Tunhou;Cheng, Dehua;He, Yuchen;Chen, Zhengxing;Dai, Xiaoliang;Xiong, Liang;Yan, Feng;Li, Hai;Chen, Yiran;Wen, Wei
深度神经网络的兴起为优化推荐系统提供了新的机会。然而，使用深度神经网络优化推荐系统需要精细的架构制造。我们提出了 NASRec，一种训练单个超网并通过权重共享有效生成丰富模型/子架构的范例。为了克服推荐领域中的数据多模态和架构异构性挑战，NASRec 建立了一个大型超网（即搜索空间）来搜索完整架构。超级网结合了运营商的多种选择和密集的连接，以最大限度地减少人类寻找先验的努力。 NASRec 的规模和异构性带来了一些挑战，例如训练效率低下、操作员不平衡和排名相关性下降。我们通过提出单操作员任意连接采样、操作员平衡交互模块和训练后微调来应对这些挑战。我们精心设计的模型 NASRecNet 在三个点击率 (CTR) 预测基准上显示出有希望的结果，表明 NASRec 的性能优于手动设计的模型和现有 NAS 方法，具有最先进的性能。我们的工作可在此处公开。
The rise of deep neural networks offers new opportunities in optimizing recommender systems. However, optimizing recommender systems using deep neural networks requires delicate architecture fabrication. We propose NASRec, a paradigm that trains a single supernet and efficiently produces abundant models/sub-architectures by weight sharing. To overcome the data multi-modality and architecture heterogeneity challenges in the recommendation domain, NASRec establishes a large supernet (i.e., search space) to search the full architectures. The supernet incorporates versatile choice of operators and dense connectivity to minimize human efforts for finding priors. The scale and heterogeneity in NASRec impose several challenges, such as training inefficiency, operator-imbalance, and degraded rank correlation. We tackle these challenges by proposing single-operator any-connection sampling, operator-balancing interaction modules, and post-training fine-tuning. Our crafted models, NASRecNet, show promising results on three Click-Through Rates (CTR) prediction benchmarks, indicating that NASRec outperforms both manually designed models and existing NAS methods with state-of-the-art performance. Our work is publicly available here.