Word segmentation from noisy data with minimal supervision

在最少的监督下从噪声数据中进行分词

• 批准号: EP/H050442/1
• 负责人: Sharon Goldwater
• 金额: $ 35.9万
• 依托单位: University of Edinburgh
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2011
• 资助国家: 英国
• 起止时间: 2011 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FH050442%2F1
• 关键词: Word; segmentation; noisy; data; minimal

In recent years, the field of natural language processing (NLP) has made great advances in a wide range of areas, such as machine translation, document summarization, and topic identification. However, much of this success is due to systems that are built using large quantities of human-annotated data in a supervised machine learning approach. This means that languages with fewer annotated resources (low-density languages) are left without much useful language technology. An important direction in NLP research is therefore to improve our ability to develop successful systems using as little annotated data as possible. Research on completely unsupervised systems is particularly interesting not only for its potential to broaden the reach of NLP technology, but also because it may shed light on the ways in which human infants manage to learn language with little or no explicit instruction.We propose to focus on the particular problem of word segmentation, and to develop a new type of probabilistic model, the infinite noisy channel model, for solving this problem in settings where little or no annotated data is available. Word segmentation refers to the problem of identifying word boundaries in either text or speech. It arises in NLP systems for many Asian languages, where words are not separated by whitespace, and also for infants learning language, because most spoken words are not separated by pauses. Previous work on unsupervised word segmentation has assumed that every time a particular word occurs, it is realized in exactly the same way. However, this is not the case for infants learning language (since words are subject to phonetic variability and noise in pronunciation), nor is it always true in NLP (if the input text contains errors, such as those produced by an optical character recognition system). Our new model will address this shortcoming by simultaneously performing word segmentation and correction of noise and variability, to recover a sequence of de-noised words from the unsegmented noisy input. We plan to develop two different versions of our model. One of these will be designed to correct for phonetic variability, and will be evaluated as a cognitive model of human language acquisition. With this model, we hope to gain insight into the computational mechanisms that allow infants to successfully extract words from noisy input, and in particular to show that the Bayesian inference techniques used in our model are a plausible explanation of infants' learning behavior. The second version of our model will be designed to correct for errors resulting from optical character recognition, and will be evaluated as a word segmentation and error-correcting NLP application in several different languages. We hope to show that the model reduces the number of character errors in the document while also producing successful segmentations. We expect these improvements to be particularly pronounced in low-density language situations.

近年来，自然语言处理（NLP）领域在机器翻译、文档摘要和主题识别等广泛领域取得了巨大进展。然而，这种成功在很大程度上是由于系统是在监督机器学习方法中使用大量人类注释的数据构建的。这意味着注释资源较少的语言（低密度语言）没有太多有用的语言技术。因此，NLP研究的一个重要方向是提高我们使用尽可能少的注释数据开发成功系统的能力。对完全无监督系统的研究特别有趣，不仅因为它有可能扩大NLP技术的范围，而且因为它可能揭示人类婴儿在很少或没有明确指导的情况下学习语言的方式。我们建议专注于分词的特殊问题，并开发一种新型的概率模型，无限噪声信道模型，用于在很少或没有注释数据可用的设置中解决该问题。分词是指在文本或语音中识别单词边界的问题。它出现在许多亚洲语言的NLP系统中，其中单词不被空格分隔，也出现在学习语言的婴儿中，因为大多数口语单词不被停顿分隔。之前关于无监督分词的工作假设每次出现特定单词时，都会以完全相同的方式实现。然而，对于学习语言的婴儿来说，情况并非如此（因为单词会受到语音变化和发音噪音的影响），在NLP中也不总是如此（如果输入文本包含错误，例如由光学字符识别系统产生的错误）。我们的新模型将通过同时执行单词分割和噪声和可变性校正来解决这个缺点，以便从未分割的噪声输入中恢复去噪单词序列。我们计划开发两个不同版本的模型。其中之一将被设计用于纠正语音变异，并将作为人类语言习得的认知模型进行评估。有了这个模型，我们希望深入了解的计算机制，让婴儿成功地从嘈杂的输入提取的话，特别是要表明，贝叶斯推理技术在我们的模型中使用的是一个合理的解释婴儿的学习行为。我们的模型的第二个版本将被设计为纠正光学字符识别导致的错误，并将作为几种不同语言的分词和纠错NLP应用程序进行评估。我们希望表明，该模型减少了文档中的字符错误的数量，同时也产生了成功的分割。我们希望这些改进在低密度语言情况下特别明显。

项目成果

Fully unsupervised small-vocabulary speech recognition using a segmental Bayesian model

• DOI: 10.21437/interspeech.2015-239
• 发表时间: 2015
• 作者: H. Kamper;A. Jansen;S. Goldwater

Unsupervised Word Segmentation and Lexicon Discovery Using Acoustic Word Embeddings

• DOI: 10.1109/taslp.2016.2517567
• 发表时间: 2016-03
• 期刊: IEEE/ACM Transactions on Audio, Speech, and Language Processing
• 作者: H. Kamper;A. Jansen;S. Goldwater

Bootstrapping a Unified Model of Lexical and Phonetic Acquisition

引导词汇和语音习得的统一模型

• 作者: Micha Elsner (Author)

A segmental framework for fully-unsupervised large-vocabulary speech recognition

• DOI: 10.1016/j.csl.2017.04.008
• 发表时间: 2016-06
• 期刊: ArXiv
• 作者: H. Kamper;A. Jansen;S. Goldwater

Weak semantic context helps phonetic learning in a model of infant language acquisition

• DOI: 10.3115/v1/p14-1101
• 发表时间: 2014-06
• 作者: Stella Frank;Naomi H Feldman;S. Goldwater