Exploring the Feasibility of Transformer Based Models on Question Relatedness
Exploring the Feasibility of Transformer Based Models on Question Relatedness
复制标题
DOI：
10.1109/hpcc-dss-smartcity-dependsys57074.2022.00136
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Honglin Shu;Pei Gao;Ziwei Yang;Chen Li;Man Wu
中科院分区：
文献类型：
--
作者：
Honglin Shu;Pei Gao;Ziwei Yang;Chen Li;Man Wu
专业的问答社区，如Stack Overflow，正在成为许多智力努力的重要方面。因此，制定快速定位相关问题和答案的策略可以有效地帮助专家解决问题。已经提出了各种检测模型来解决与问题相关的预测问题，但都是次优的，因为它们不能有效地捕捉长序列的长距离相关性。随着自我注意机制和转换器的发明，一种处理与问题相关的预测的更好的方法刚刚出现。本研究的主要目的是探讨基于转换的问题相关问题模型的可行性。我们将问题相关度问题转化为文本分类问题，并引入了一个具有代表性的基于转换器的模型，即双向编码器Transformers表示(BERT)来处理文本分类。在我们的实验中，BERT的性能优于SoftSVM和BiDotLSTM，这意味着基于转换器的模型在解决与问题相关的问题方面具有相当大的潜力。此外，由于我们可以将文本分类问题重新表述为链接预测问题，因此我们讨论了将基于变换的方法和图表示学习相结合的可能性。
Professional question answering communities, such as Stack Overflow, are becoming a significant aspect of many intellectual endeavors. As a result, developing a strategy for swiftly locating relevant questions and answers can effectively assist experts in issue solving. Various detection models have been presented to address the problem of question-relatedness prediction, but all are sub-optimal since they cannot effectively capture the long-distance dependency of a long sequence. With the invention of the self-attention mechanism and transformer, a better approach to dealing with question-relatedness prediction has just emerged. The primary objective of this study is to investigate the feasibility of the transformer-based model for question-relatedness problems. We turn the question relatedness problem into a text classification problem and introduce a representative transformer-based model, namely Bidirectional Encoder Representation from Transformers (BERT), to deal with text classification. In our experiment, we show that BERT outperforms both SoftSVM and BiDotLSTM, implying that the transformer-based model has considerable potential to address the challenge in the question-relatedness problem. Furthermore, since we can re-formulate text classification problem into a link prediction problem, we discuss the possibility of incorporating the transformer-based approach and graph representation learning.