Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer
Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer
复制标题
DOI：
10.1609/aaai.v36i10.21410
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Wenda Xu;Michael Stephen Saxon;Misha Sra;W. Wang
中科院分区：
文献类型：
--
作者：
Wenda Xu;Michael Stephen Saxon;Misha Sra;W. Wang
专家-外行文本风格转换技术有可能改善科学界成员与公众之间的沟通。专家提供的高质量信息往往充满了外行难以理解的术语。这在医学领域是一个特别值得注意的问题，外行人经常被在线医学文本所迷惑。目前，有两个瓶颈阻碍了建立高质量的医学专家-外行风格的转换系统的目标：缺乏跨专家和外行术语的预训练医学领域语言模型，以及缺乏用于训练转换任务本身的平行语料库。为了缓解第一个问题，我们提出了一种新的语言模型（LM）预训练任务，知识库同化，在自监督学习过程中将专家和外行风格的医学术语图的边缘的预训练数据合成到LM中。为了缓解第二个问题，我们建立了一个大规模的平行语料库，在医学专家，外行域使用基于边缘的标准。我们的实验表明，基于transformer的模型在知识库同化和其他完善的预训练任务上进行了预训练，在我们新的并行语料库上进行了微调，导致了对专家-外行迁移基准的相当大的改进，使我们的人类评估平均相对提高了106%，总体成功率（OSR）。
Expert-layman text style transfer technologies have the potential to improve communication between members of scientific communities and the general public. High-quality information produced by experts is often filled with difficult jargon laypeople struggle to understand. This is a particularly notable issue in the medical domain, where layman are often confused by medical text online. At present, two bottlenecks interfere with the goal of building high-quality medical expert-layman style transfer systems: a dearth of pretrained medical-domain language models spanning both expert and layman terminologies and a lack of parallel corpora for training the transfer task itself. To mitigate the first issue, we propose a novel language model (LM) pretraining task, Knowledge Base Assimilation, to synthesize pretraining data from the edges of a graph of expert- and layman-style medical terminology terms into an LM during self-supervised learning. To mitigate the second issue, we build a large-scale parallel corpus in the medical expert-layman domain using a margin-based criterion. Our experiments show that transformer-based models pretrained on knowledge base assimilation and other well-established pretraining tasks fine-tuning on our new parallel corpus leads to considerable improvement against expert-layman transfer benchmarks, gaining an average relative improvement of our human evaluation, the Overall Success Rate (OSR), by 106%.