Improvement of topic-based language models using Dirichlet mixtures and their applications

点击翻译按钮获取中文摘要

For improving statistical language models, we enhanced predictive power of ngram models, which are typical language models, using topic or context information. We proposed new estimation methods for Dirichlet mixtures and evaluated the model on applications ; speech recognition and statistical machine translation.1. We developed a robust estimation method for Dirichlet mixtures language models using hierarchical Bayesian models. In order to approximate integration appeared in Bayesian inference, we used the reversing-EM and variational approximation. In the experiments using various text data, we showed the estimation method achieves the lowest perplexity level.2. Our model was integrated in speech recognition systems, and evaluated by recognition rate. Two integration methods were developed ; (1) modification of probability of trigram models using the unigram rescaling, (2) optimization on document level using document likelihood computed by our model. Comparing Latent Dirichlet Allocation (LDA) with our model, we showed the speech recognition rate of the system with our model is higher than that of LDA.3. We proposed cross-language Dirichlet mixture models which were integrated in phrase-based statistical machine translation systems. Using this model, the system can select contextually or topically correct Japanese words from candidates as translation of English input document. Experiments using newspaper articles translation showed that topic models were effective for lower perplexity.