Evaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representations
Evaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representations
复制标题
DOI：
10.18653/v1/d19-1060
复制
发表时间：
2019-08
期刊：
影响因子：
--
通讯作者：
Mingda Chen;Zewei Chu;Kevin Gimpel
中科院分区：
文献类型：
--
作者：
Mingda Chen;Zewei Chu;Kevin Gimpel
先前关于预训练句子嵌入和基准的工作集中在独立句子的能力上。我们提出了DiscoEval，一个测试套件的任务，以评估句子表示是否包括更广泛的上下文信息。我们还提出了各种训练目标，利用维基百科的自然注释来构建能够对话语进行建模的句子编码器。我们对使用我们提出的训练目标进行预训练的句子编码器以及其他流行的预训练句子编码器在DiscoEval和其他句子评估任务上进行了基准测试。从经验上讲，我们表明，这些培训目标有助于编码不同方面的信息在文档结构。此外，BERT和埃尔莫表现出较强的性能与DiscoEval的个人隐藏层显示出不同的特点。
Prior work on pretrained sentence embeddings and benchmarks focus on the capabilities of stand-alone sentences. We propose DiscoEval, a test suite of tasks to evaluate whether sentence representations include broader context information. We also propose a variety of training objectives that makes use of natural annotations from Wikipedia to build sentence encoders capable of modeling discourse. We benchmark sentence encoders pretrained with our proposed training objectives, as well as other popular pretrained sentence encoders on DiscoEval and other sentence evaluation tasks. Empirically, we show that these training objectives help to encode different aspects of information in document structures. Moreover, BERT and ELMo demonstrate strong performances over DiscoEval with individual hidden layers showing different characteristics.