Early Exiting BERT for Efficient Document Ranking
Early Exiting BERT for Efficient Document Ranking
复制标题
DOI：
10.18653/v1/2020.sustainlp-1.11
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Ji Xin;Rodrigo Nogueira;Yaoliang Yu;Jimmy J. Lin
中科院分区：
文献类型：
--
作者：
Ji Xin;Rodrigo Nogueira;Yaoliang Yu;Jimmy J. Lin
预先训练的语言模型，如BERT，已经在各种任务中表现出了它们的有效性。尽管它们很强大，但它们被认为是计算密集型的，这阻碍了现实世界的应用。在本文中，我们介绍了早期退出BERT的文档排序。稍加修改，BERT就变成了一个具有多个输出路径的模型，每个推理样本都可以从这些路径中提前退出。通过这种方式，可以在样本之间有效地分配计算，并且在保持原始质量的同时显著降低了整体系统延迟。我们在两个文档排名数据集上的实验表明，推理速度高达2.5倍，质量下降最小。我们实现的源代码可以在https://github.com/castorini/earlyexiting-monobert上找到。
Pre-trained language models such as BERT have shown their effectiveness in various tasks. Despite their power, they are known to be computationally intensive, which hinders real-world applications. In this paper, we introduce early exiting BERT for document ranking. With a slight modification, BERT becomes a model with multiple output paths, and each inference sample can exit early from these paths. In this way, computation can be effectively allocated among samples, and overall system latency is significantly reduced while the original quality is maintained. Our experiments on two document ranking datasets demonstrate up to 2.5x inference speedup with minimal quality degradation. The source code of our implementation can be found at https://github.com/castorini/earlyexiting-monobert.