A Generalized Framework for Hierarchical Word Sequence Language Model
A Generalized Framework for Hierarchical Word Sequence Language Model
复制标题
DOI：
--
复制
发表时间：
2016-10
期刊：
影响因子：
--
通讯作者：
Xiaoyi Wu;Kevin Duh;Yuji Matsumoto
中科院分区：
文献类型：
--
作者：
Xiaoyi Wu;Kevin Duh;Yuji Matsumoto
语言建模是一个基础研究问题，在任何自然语言处理任务中都有广泛的应用。为了估计自然语言句子的概率，大多数语言建模研究都使用基于n-gra的方法来考虑句子的概率。然而，n-gra模型下的假设不够鲁棒，无法解决数据稀疏性问题，从而影响了语言模型的最终性能。在这一点上，层次词序列（Hierarchical Word Sequence，缩写为HWS）语言模型可以被视为一种有效的替代方法。在本文中，我们将HWS - m模型推广到一个框架中，在这个框架中，可以采用不同的假设以完全无监督的方式对词序列进行重排，这大大增加了HWS - m模型的可扩展性。为了评估，我们将重新排列的单词序列与传统的n-gra m单词序列进行比较。内部和外部实验都验证了我们的网络可以达到更好的性能，证明我们的m方法可以被认为是n-gra m语言m模型的更好选择。
Language m odeling is a funda m ental research proble m that has wide application for m any NLP tasks. For esti m ating probabilities of nat-ural language sentences, m ost research on language m odeling use n-gra m based approaches to factor sentence probabilities. However, the assu m ption under n-gra m m odels is not robust enough to cope with the data sparseness prob-le m , which affects the final perfor m ance of language m odels. At the point, Hierarchical Word Sequence (abbreviated as HWS) language m odels can be viewed as an effective alternative to nor m al n-gra m m ethod. In this paper, we generalize HWS m odels into a fra m ework, where different assu m ptions can be adopted to rearrange word sequences in a totally unsupervised fashion, which greatly increases the expandability of HWS m odels. For evaluation, we co m pare our rearranged word sequences to conventional n-gra m word sequences. Both intrinsic and extrinsic exper-i m ents verify that our fra m ework can achieve better perfor m ance, proving that our m ethod can be considered as a better alternative for n-gra m language m odels.