BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition
BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition
复制标题
DOI：
10.18653/v1/2021.acl-long.482
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Yinghao Li;Pranav Shetty;Lu Liu;Chao Zhang;Le Song
中科院分区：
文献类型：
--
作者：
Yinghao Li;Pranav Shetty;Lu Liu;Chao Zhang;Le Song
我们研究了从多个弱监督源中利用噪声标签学习命名实体识别(NER)标记器的问题。虽然获取成本较低，但来自弱监管来源的标签往往是不完整、不准确和相互矛盾的，这使得学习准确的NER模型变得困难。为了解决这一挑战，我们提出了一种条件隐马尔可夫模型(CHMM)，该模型可以在无监督的情况下从多源噪声标签中有效地推断出真实标签。CHMM利用预先训练的语言模型的上下文表示能力，改进了经典的隐马尔可夫模型。具体地说，CHMM从输入令牌的BERT嵌入中学习令牌的逐级转移和发射概率，以从噪声观测中推断潜在的真标签。我们用一种交替训练方法(CHMM-ALT)进一步提炼CHMM。它用CHMM推断的标签微调Bert-ner模型，该Bert-ner的输出被视为训练CHMM的额外弱源。在四个不同领域的NER基准测试上的实验表明，我们的方法比最先进的弱监督NER模型有很大的优势。
We study the problem of learning a named entity recognition (NER) tagger using noisy labels from multiple weak supervision sources. Though cheap to obtain, the labels from weak supervision sources are often incomplete, inaccurate, and contradictory, making it difficult to learn an accurate NER model. To address this challenge, we propose a conditional hidden Markov model (CHMM), which can effectively infer true labels from multi-source noisy labels in an unsupervised way. CHMM enhances the classic hidden Markov model with the contextual representation power of pre-trained language models. Specifically, CHMM learns token-wise transition and emission probabilities from the BERT embeddings of the input tokens to infer the latent true labels from noisy observations. We further refine CHMM with an alternate-training approach (CHMM-ALT). It fine-tunes a BERT-NER model with the labels inferred by CHMM, and this BERT-NER’s output is regarded as an additional weak source to train the CHMM in return. Experiments on four NER benchmarks from various domains show that our method outperforms state-of-the-art weakly supervised NER models by wide margins.