Improving the robustness and accuracy of biomedical language models through adversarial training
Improving the robustness and accuracy of biomedical language models through adversarial training
复制标题
DOI：
10.1016/j.jbi.2022.104114
复制
发表时间：
2022-08-01
影响因子：
4.5
通讯作者：
Samwald, Matthias
中科院分区：
文献类型：
--
作者：
Moradi, Milad;Samwald, Matthias
深度Transformer神经网络模型提高了生物医学领域智能文本处理系统的预测精度。他们在各种生物医学和临床自然语言处理（NLP）基准上获得了最先进的性能分数。然而，这些模型的鲁棒性和可靠性迄今为止还没有得到充分的研究。神经NLP模型很容易被对抗性样本欺骗，即对输入进行微小的更改，保留文本的含义和可理解性，但迫使NLP系统做出错误的决定。这引起了人们对生物医学NLP系统的安全性和可信度的严重担忧，特别是当它们打算部署在现实世界的用例中时。我们研究了几个Transformer神经语言模型的鲁棒性，即BioBERT，SciBERT，BioMed-RoBERTa和Bio-ClinicalBERT，在广泛的生物医学和临床文本处理任务。我们实现了各种对抗性攻击方法来测试不同攻击场景下的NLP系统。实验结果表明，生物医学NLP模型对对抗性样本敏感;在Micro-F1、Pearson相关性和准确性指标上，它们在字符级和单词级对抗性噪声上的性能平均下降了21%和18.9%。通过进行广泛的对抗性训练实验，我们在干净样本和对抗性输入的混合上微调了NLP模型。结果表明，对抗性训练是对抗性噪声的有效防御机制;模型的鲁棒性平均提高了11.3%。此外，模型在干净数据上的性能平均提高了2.4%，这表明对抗性训练可以提高生物医学NLP系统的泛化能力。这项研究朝着揭示深度神经语言模型在生物医学NLP应用中的脆弱性迈出了重要一步。它还为在生物医学领域开发安全、可信和准确的智能文本处理系统提供了实用有效的策略。
Deep transformer neural network models have improved the predictive accuracy of intelligent text processing systems in the biomedical domain. They have obtained state-of-the-art performance scores on a wide variety of biomedical and clinical Natural Language Processing (NLP) benchmarks. However, the robustness and reliability of these models has been less explored so far. Neural NLP models can be easily fooled by adversarial samples, i.e. minor changes to input that preserve the meaning and understandability of the text but force the NLP system to make erroneous decisions. This raises serious concerns about the security and trust-worthiness of biomedical NLP systems, especially when they are intended to be deployed in real-world use cases. We investigated the robustness of several transformer neural language models, i.e. BioBERT, SciBERT, BioMed-RoBERTa, and Bio-ClinicalBERT, on a wide range of biomedical and clinical text processing tasks. We implemented various adversarial attack methods to test the NLP systems in different attack scenarios. Experimental results showed that the biomedical NLP models are sensitive to adversarial samples; their performance dropped in average by 21 and 18.9 absolute percent on character-level and word-level adversarial noise, respectively, on Micro-F1, Pearson Correlation, and Accuracy measures. Conducting extensive adversarial training experiments, we fine-tuned the NLP models on a mixture of clean samples and adversarial inputs. Results showed that adversarial training is an effective defense mechanism against adversarial noise; the models' robustness improved in average by 11.3 absolute percent. In addition, the models' performance on clean data increased in average by 2.4 absolute percent, demonstrating that adversarial training can boost generalization abilities of biomedical NLP systems. This study takes an important step towards revealing vulnerabilities of deep neural language models in biomedical NLP applications. It also provides practical and effective strategies to develop secure, trust-worthy, and accurate intelligent text processing systems in the biomedical domain.