Trigger-Based Language Model Construction by Combining Different Corpora (第6回音声言語シンポジウム)
Trigger-Based Language Model Construction by Combining Different Corpora (第6回音声言語シンポジウム)
复制标题
基于触发的结合不同语料库的语言模型构建（第六届口语研讨会）
DOI：
--
复制
发表时间：
2004
期刊：
影响因子：
--
通讯作者：
山本 博史
中科院分区：
文献类型：
--
作者：
Carlos Troncoso;河原 達也;山本 博史
本文研究了基于触发器的语言模型，该模型可以比n-gram语言模型建模的词之间的依赖关系更长。通常在语言建模中，当训练语料库与目标任务匹配时，其大小通常很小，因此不足以提供可靠的概率估计。另一方面，大型语料库通常过于一般化，无法捕获任务依赖性。该方法试图通过构建基于触发器的语言模型来克服这种一般性和稀疏性的权衡问题，该模型首先从匹配任务的语料库中提取任务相关的触发对，然后从任务语料库和大型文本语料库中估计这些触发对的出现概率，以避免数据稀疏性问题。我们报告了自发性日语语料库（CSJ）的评价结果。
In this paper we study the trigger-based language model, which can model dependencies between words longer than those modeled by the n-gram language model. Generally in language modeling, when the training corpus matches the target task, its size is typically small, and therefore insufficient for providing reliable probability estimates. On the other hand, large corpora are often too general to capture task dependency. The proposed approach tries to overcome this generality-sparseness trade-off problem by constructing a trigger-based language model in which task-dependent trigger pairs are first extracted from the corpus that matches the task, and then the occurrence probabilities of the pairs are estimated from both the task corpus and a large text corpus to avoid the data sparseness problem. We report evaluation results in the Corpus of Spontaneous Japanese (CSJ).
DOI：
--
发表时间：
2004
期刊：
North American Chapter of the Association for Computational Linguistics
影响因子：
--
作者：
Nobuhiro Kaji;Masashi Okamoto;S. Kurohashi
通讯作者：
S. Kurohashi