Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts
Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts
复制标题
DOI：
10.1145/3539597.3570475
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Yu Zhang;Yunyi Zhang;Martin Michalski;Yucheng Jiang;Yu Meng;Jiawei Han
中科院分区：
文献类型：
--
作者：
Yu Zhang;Yunyi Zhang;Martin Michalski;Yucheng Jiang;Yu Meng;Jiawei Han
种子引导的主题发现方法不是以完全无监督的方式从给定的文本语料库中挖掘连贯的主题，而是利用用户提供的种子词来提取独特且连贯的主题，使得挖掘出的主题可以更好地迎合用户的兴趣。为了对单词和种子之间的语义相关性进行建模以发现主题指示术语，现有的种子引导方法利用不同类型的上下文信号，例如文档级单词共现、基于滑动窗口的局部上下文以及由预训练的语言模型带来的通用语言学知识。在这项工作中，我们分析并经验地表明，每种类型的上下文信息在种子指导下建模单词语义时都有其价值和局限性，但结合三种类型的上下文（即，从局部上下文学习的词嵌入、从通用域训练获得的预训练语言模型表示、以及基于种子信息检索的主题指示语句）允许它们彼此互补以发现高质量的主题。我们提出了一个迭代框架SeedTopicMine，它联合学习三种类型的上下文，并通过集成排名过程逐渐融合它们的上下文信号。在不同的种子集和多个数据集上，SeedTopicMine始终比现有的种子引导主题发现方法产生更一致和准确的主题。
Instead of mining coherent topics from a given text corpus in a completely unsupervised manner, seed-guided topic discovery methods leverage user-provided seed words to extract distinctive and coherent topics so that the mined topics can better cater to the user's interest. To model the semantic correlation between words and seeds for discovering topic-indicative terms, existing seed-guided approaches utilize different types of context signals, such as document-level word co-occurrences, sliding window-based local contexts, and generic linguistic knowledge brought by pre-trained language models. In this work, we analyze and show empirically that each type of context information has its value and limitation in modeling word semantics under seed guidance, but combining three types of contexts (i.e., word embeddings learned from local contexts, pre-trained language model representations obtained from general-domain training, and topic-indicative sentences retrieved based on seed information) allows them to complement each other for discovering quality topics. We propose an iterative framework, SeedTopicMine, which jointly learns from the three types of contexts and gradually fuses their context signals via an ensemble ranking process. Under various sets of seeds and on multiple datasets, SeedTopicMine consistently yields more coherent and accurate topics than existing seed-guided topic discovery approaches.