Language Models
Language Models
复制标题
DOI：
10.1007/978-0-387-39940-9_923
复制
发表时间：
2009
影响因子：
0.5
通讯作者：
Jordan L. Boyd-Graber;Philipp Koehn
中科院分区：
文献类型：
--
作者：
Jordan L. Boyd-Graber;Philipp Koehn
定义语言模型根据一些训练数据将概率分配给一段看不见的文本。例如，基于大型英文报纸档案的语言模型预计会将更高的概率分配给“一点文本”而不是“AW PIT TOV标签”，因为前一短语中的单词(或者如果使用所谓的N元语法模型，则为单词对或单词三元组)在数据中出现的频率比后一短语中的单词更频繁。对于信息检索，典型的用法是为每个文档构建一个语言模型。在搜索时，排名最靠前的文档是语言模型赋予查询最高概率的文档。历史背景语言模型一词起源于20世纪80年代初为自动语音识别系统开发的语言生成的概率模型[9]。语音识别系统使用语言模型来补充声学模型的结果，该声学模型对单词(或称为音素的单词的一部分)和声学信号之间的关系进行建模。然而，语言模型的历史可以追溯到20世纪初，当时安德烈·马尔科夫在俄罗斯文学作品中使用语言模型(马尔可夫模型)来模拟字母序列[3]。语言模型的另一个著名应用是克劳德·香农的字母序列和单词序列模型，他用它来说明编码和信息理论的含义。在20世纪90年代，S语言模型被用作几个自然语言处理应用的通用工具，如词性标注、机器翻译和光学字符识别。语言模型在20世纪90年代末被一些研究小组应用于信息检索，S[4，7，14，15]。它们在信息检索研究中迅速流行起来。到2001年，ACM SIGIR会议有两个关于语言模式的单独会议，总共包含5篇论文[13]。2003年，一批领先的信息检索研究人员发布了一份研究路线图《信息检索和语言建模的挑战》[1]，表明信息检索的未来和语言建模的未来是密不可分的。
DEFINITION A language model assigns a probability to a piece of unseen text, based on some training data. For example, a language model based on a big English newspaper archive is expected to assign a higher probability to “a bit of text” than to “aw pit tov tags”, because the words in the former phrase (or word pairs or word triples if so-called N-GRAM MODELS are used) occur more frequently in the data than the words in the latter phrase. For information retrieval, typical usage is to build a language model for each document. At search time, the top ranked document is the one which’language model assigns the highest probability to the query.HISTORICAL BACKGROUND The term language models originates from probabilistic models of language generation developed for automatic speech recognition systems in the early 1980’s [9]. Speech recognition systems use a language model to complement the results of the acoustic model which models the relation between words (or parts of words called phonemes) and the acoustic signal. The history of language models, however, goes back to beginning of the 20th century when Andrei Markov used language models (Markov models) to model letter sequences in works of Russian literature [3]. Another famous application of language models are Claude Shannon’s models of letter sequences and word sequences, which he used to illustrate the implications of coding and information theory [17]. In the 1990’s language models were applied as a general tool for several natural language processing applications, such as part-of-speech tagging, machine translation, and optical character recognition. Language models were applied to information retrieval by a number of research groups in the late 1990’s [4, 7, 14, 15]. They became rapidly popular in information retrieval research. By 2001, the ACM SIGIR conference had two separate sessions on language models containing 5 papers in total [13]. In 2003, a group of leading information retrieval researchers published a research roadmap “Challenges in Information Retrieval and Language Modeling”[1], indicating that the future of information retrieval and the future of language modeling can not be seen apart from each other.