A Comprehensive Survey on Word Representation Models: From Classical to State-of-the-Art Word Representation Language Models
A Comprehensive Survey on Word Representation Models: From Classical to State-of-the-Art Word Representation Language Models
复制标题
DOI：
10.1145/3434237
复制
发表时间：
2021-09-01
影响因子：
2
通讯作者：
Prasad, Mukesh
中科院分区：
文献类型：
--
作者：
Naseem, Usman;Razzak, Imran;Prasad, Mukesh
词表示一直是自然语言处理（NLP）的一个重要研究领域。理解如此复杂的文本数据是必要的，因为它包含丰富的信息，可以在各种应用程序中广泛使用。在本调查中，我们探索了不同的单词表示模型及其表达能力，从经典到现代最先进的单词表示语言模型（LMS）。我们描述了各种文本表示方法，模型设计在NLP的背景下蓬勃发展，包括SOTA LM。这些模型可以将大量文本转换为有效的矢量表示，捕获相同的语义信息。此外，这种表示可以由各种机器学习（ML）算法用于各种NLP相关任务。最后，本调查简要讨论了常用的基于ML和DL的分类器，评估指标，以及这些词嵌入在不同NLP任务中的应用。
Word representation has always been an important research area in the history of natural language processing (NLP). Understanding such complex text data is imperative, given that it is rich in information and can be used widely across various applications. In this survey, we explore different word representation models and its power of expression, from the classical to modern-day state-of-the-art word representation language models (LMS). We describe a variety of text representation methods, and model designs have blossomed in the context of NLP, including SOTA LMs. These models can transform large volumes of text into effective vector representations capturing the same semantic information. Further, such representations can be utilized by various machine learning (ML) algorithms for a variety of NLP-related tasks. In the end, this survey briefly discusses the commonly used ML- and DL-based classifiers, evaluation metrics, and the applications of these word embeddings in different NLP tasks.