SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features

SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features
复制标题

DOI：
--
复制
发表时间：
2022-06
期刊：
--
影响因子：
--
通讯作者：
J. Opitz;A. Frank
J. Opitz;A. Frank
中科院分区：
其他
文献类型：
--
作者：
J. Opitz;A. Frank

文献摘要

被引文献

相似文献

基于大型预训练语言模型的模型，例如 S(entence)BERT，提供了有效且高效的句子嵌入，与人类相似度评级具有高度相关性，但缺乏可解释性。另一方面，基于图的含义表示（例如，抽象含义表示，AMR）的图度量可以明确两个句子相似的语义方面。然而，此类指标往往很慢，依赖于解析器，并且在评估句子相似性时无法达到最先进的性能。在这项工作中，我们的目标是通过学习诱导语义结构化句子 BERT 嵌入（S^3BERT）来实现两全其美。我们的 S^3BERT 嵌入由可解释的子嵌入组成，强调各种句子含义特征（例如语义角色、否定或量化）。我们展示了如何 i) 通过近似一组可解释的语义 AMR 图度量来学习将句子嵌入分解为意义特征，以及如何 ii) 通过控制分解学习过程来保持神经嵌入的整体能力，第二个目标是强制与 SBERT 教师模型的相似性评级保持一致。在我们的实验研究中，我们表明我们的方法提供了可解释性，同时保留了神经句子嵌入的有效性和效率。
Models based on large-pretrained language models, such as S(entence)BERT, provide effective and efficient sentence embeddings that show high correlation to human similarity ratings, but lack interpretability. On the other hand, graph metrics for graph-based meaning representations (e.g., Abstract Meaning Representation, AMR) can make explicit the semantic aspects in which two sentences are similar. However, such metrics tend to be slow, rely on parsers, and do not reach state-of-the-art performance when rating sentence similarity. In this work, we aim at the best of both worlds, by learning to induce Semantically Structured Sentence BERT embeddings (S^3BERT). Our S^3BERT embeddings are composed of explainable sub-embeddings that emphasize various sentence meaning features (e.g., semantic roles, negation, or quantification). We show how to i) learn a decomposition of the sentence embeddings into meaning features, through approximation of a suite of interpretable semantic AMR graph metrics, and how to ii) preserve the overall power of the neural embeddings by controlling the decomposition learning process with a second objective that enforces consistency with the similarity ratings of an SBERT teacher model. In our experimental studies, we show that our approach offers interpretability – while preserving the effectiveness and efficiency of the neural sentence embeddings.