Improving Neural Topic Models Using Knowledge Distillation
Improving Neural Topic Models Using Knowledge Distillation
复制标题
DOI：
10.18653/v1/2020.emnlp-main.137
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Alexander Miserlis Hoyle;Pranav Goel;P. Resnik
中科院分区：
文献类型：
--
作者：
Alexander Miserlis Hoyle;Pranav Goel;P. Resnik
主题模型通常用于识别人类可解释的主题，以帮助理解大型文档集合。我们使用知识蒸馏将概率主题模型和预训练的变压器的最佳属性结合起来。我们的模块化方法可以直接应用于任何神经主题模型来提高主题质量，我们使用两个具有不同架构的模型来证明这一点，获得了最先进的主题一致性。我们表明，我们的适应性框架不仅提高了所有估计主题的总体性能，正如通常报道的那样，而且还提高了对齐主题的正面比较。
Topic models are often used to identify human-interpretable topics to help make sense of large document collections. We use knowledge distillation to combine the best attributes of probabilistic topic models and pretrained transformers. Our modular method can be straightforwardly applied with any neural topic model to improve topic quality, which we demonstrate using two models having disparate architectures, obtaining state-of-the-art topic coherence. We show that our adaptable framework not only improves performance in the aggregate over all estimated topics, as is commonly reported, but also in head-to-head comparisons of aligned topics.