Multi-Faceted Knowledge-Driven Pre-Training for Product Representation Learning
Multi-Faceted Knowledge-Driven Pre-Training for Product Representation Learning
复制标题
DOI：
10.1109/tkde.2022.3200921
复制
发表时间：
2023-07
影响因子：
8.9
通讯作者：
Denghui Zhang;Yanchi Liu;Zixuan Yuan;Yanjie Fu;Haifeng Chen;Hui Xiong
中科院分区：
文献类型：
--
作者：
Denghui Zhang;Yanchi Liu;Zixuan Yuan;Yanjie Fu;Haifeng Chen;Hui Xiong
作为电子商务计算的关键组件，产品表示学习（PRL）为各种应用程序提供了好处，包括产品匹配、搜索和分类。现有的PRL方法由于无法捕获上下文化语义，导致其语言理解能力较差。此外，通过现有方法学习到的表示不容易转移到新产品中。受预训练语言模型（plm）最新进展的启发，我们尝试为PRL调整plm以缓解上述问题。在本文中，我们开发了一个知识驱动的产品表示学习预训练框架KINDLE，它可以稳健灵活地保留上下文语义和多面产品知识。具体而言，我们首先将传统的一阶段预训练扩展到两阶段预训练框架，并利用有意识的知识编码器来确保知识顺利融合到PLM中。此外，我们提出了一种多目标异构嵌入方法来表示数千个知识元素。这有助于KINDLE通过在知识获取任务中取代孤立的类作为训练目标来自动校准知识噪声和稀疏性。此外，提出了一种输入感知的门控网络，为不同的下游任务选择最相关的知识。最后，大量的实验证明了KINDLE在三个下游任务的最先进的基线上的优势。
As a key component of e-commerce computing, product representation learning (PRL) provides benefits for a variety of applications, including product matching, search, and categorization. The existing PRL approaches have poor language understanding ability due to their inability to capture contextualized semantics. In addition, the learned representations by existing methods are not easily transferable to new products. Inspired by the recent advance of pre-trained language models (PLMs), we make the attempt to adapt PLMs for PRL to mitigate the above issues. In this article, we develop KINDLE, a Knowledge-drIven pre-trainiNg framework for proDuct representation LEarning, which can preserve the contextual semantics and multi-faceted product knowledge robustly and flexibly. Specifically, we first extend traditional one-stage pre-training to a two-stage pre-training framework, and exploit a deliberate knowledge encoder to ensure a smooth knowledge fusion into PLM. In addition, we propose a multi-objective heterogeneous embedding method to represent thousands of knowledge elements. This helps KINDLE calibrate knowledge noise and sparsity automatically by replacing isolated classes as training targets in knowledge acquisition tasks. Furthermore, an input-aware gating network is proposed to select the most relevant knowledge for different downstream tasks. Finally, extensive experiments have demonstrated the advantages of KINDLE over the state-of-the-art baselines across three downstream tasks.