Paraphrase Identification with Lexical, Syntactic and Sentential Encodings
Paraphrase Identification with Lexical, Syntactic and Sentential Encodings
复制标题
DOI：
10.3390/app10124144
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Sheng Xu;Xing Shen;Fumiyo Fukumoto;Jiyi Li;Yoshimi Suzuki;H. Nishizaki
中科院分区：
文献类型：
--
作者：
Sheng Xu;Xing Shen;Fumiyo Fukumoto;Jiyi Li;Yoshimi Suzuki;H. Nishizaki
释义识别一直是自然语言处理(NLP)领域的重要课题之一。然而，如何将句子中的各种语境(如词汇和语义信息)解释为相关特征仍然是一个悬而未决的问题。本文针对这一问题，提出了一种基于神经网络的学习模型来利用上下文特征的方法。我们的词汇、句法和句法编码(LSSE)学习模型结合了关系图卷积网络(R-GCN)，以利用来自本地上下文的不同特征，即单词编码、位置编码和完全依存结构。通过利用R-GCNS获得的隐含状态以及双向编码器表示(BERT)的词汇和句子编码，我们的模型有效地学习了句子之间的上下文相似性。在微软研究释义语料库(MRPC)和Quora问题对(QQP)两个基准数据集上的实验结果表明，与基准的Bert句子编码模型相比，MRPC上的F1分数提高了1.7%，QQP上的F1分数提高了1.0%。此外，我们还验证了位置编码和句法特征的结合有助于提高性能。
Paraphrase identification has been one of the major topics in Natural Language Processing (NLP). However, how to interpret a diversity of contexts such as lexical and semantic information within a sentence as relevant features is still an open problem. This paper addresses the problem and presents an approach for leveraging contextual features with a neural-based learning model. Our Lexical, Syntactic, and Sentential Encodings (LSSE) learning model incorporates Relational Graph Convolutional Networks (R-GCNs) to make use of different features from local contexts, i.e., word encoding, position encoding, and full dependency structures. By utilizing the hidden states obtained by the R-GCNs as well as lexical and sentential encodings by Bidirectional Encoder Representations from Transformers (BERT), our model learns the contextual similarity between sentences effectively. The experimental results by using the two benchmark datasets, Microsoft Research Paraphrase Corpus (MRPC) and Quora Question Pairs (QQP) show that the improvement compared with the baseline, BERT sentential encodings model, was 1.7% F1-score on MRPC and 1.0% F1-score on QQP. Moreover, we verified that the combination of position encoding and syntactic features contributes to performance improvement.