Language Representation Models: An Overview.
Language Representation Models: An Overview.
复制标题
DOI：
10.3390/e23111422
复制
发表时间：
2021-10-28
期刊：
影响因子：
--
通讯作者：
Tropmann-Frick M
中科院分区：
文献类型：
--
作者：
Schomacker T;Tropmann-Frick M
在过去的几十年里，文本挖掘已经被用来从自由文本中提取知识。多年来，将神经网络和深度学习应用于自然语言处理（NLP）任务已经为现实世界的语言问题带来了许多成就。过去五年的发展已经产生了允许在NLP中实际应用迁移学习的技术。该领域的进展是实质性的，并且已经实现了基于一般语言理解评估的优于人类基线性能的里程碑。本文进行了有针对性的文献综述，以概述，描述，解释，并将有助于实现这一里程碑的关键技术纳入上下文。这里介绍的研究是对神经语言模型的有针对性的回顾，这些模型向通用语言表示模型迈出了重要的一步。
In the last few decades, text mining has been used to extract knowledge from free texts. Applying neural networks and deep learning to natural language processing (NLP) tasks has led to many accomplishments for real-world language problems over the years. The developments of the last five years have resulted in techniques that have allowed for the practical application of transfer learning in NLP. The advances in the field have been substantial, and the milestone of outperforming human baseline performance based on the general language understanding evaluation has been achieved. This paper implements a targeted literature review to outline, describe, explain, and put into context the crucial techniques that helped achieve this milestone. The research presented here is a targeted review of neural language models that present vital steps towards a general language representation model.
DOI：
10.1145/3434237
发表时间：
2021-09-01
影响因子：
2
作者：
Naseem, Usman;Razzak, Imran;Prasad, Mukesh
通讯作者：
Prasad, Mukesh
影响因子：
4.9
作者：
Liu, Yinhan;Ott, Myle;Stoyanov, Veselin
通讯作者：
Stoyanov, Veselin
影响因子：
4.9
作者：
Clark, Kevin;Luong, Minh-Thang;Manning, Christopher D.
通讯作者：
Manning, Christopher D.