Automating Dictionary Construction for Better Natural Language Processing

自动构建字典以实现更好的自然语言处理

• 批准号: RGPIN-2015-05615
• 负责人: Cook, ChristopherPaul
• 金额: $ 1.68万
• 依托单位: University of New Brunswick
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=648019
• 关键词: Automating; Dictionary; Construction; Better; Natural

Language technology is an important component of many everyday software systems, such as predictive text entry on smartphones, spelling checkers in word processors, and automatic translation tools. Many language technology systems rely on dictionary-like resources that contain specialized information about words.******Language is always changing. For example, new words, such as "mansplain" and "staycation", are coined every day. Established words regularly take on new meanings, such as the new senses of "post" and "wall" related to social media. New expressions also emerge, such as "flash mob". Dictionaries must therefore be continually updated to include these new usages.******When information is missing from a language technology system's dictionary, the system won't work as well as it could otherwise. Keeping dictionaries up-to-date is therefore essential to building high quality language technology.******Dictionaries are also important tools for language learners and translators, and are valuable in terms of cultural heritage. To be most useful for these purposes, dictionaries must also be kept up-to-date.******Traditionally, lexicographers manually analyze large collections of texts to identify relevant facts about how words are used, and subsequently write dictionary entries. However, a staggering amount of text is written every day, particularly through online sources such as the web and social media. It is simply not possible for lexicographers to keep up with it all.******How then, can dictionaries be kept up-to-date?******Automatic methods are required. This research program will first consider how to best build massive document collections from the web. Drawing on recent linguistic theory and research in statistical machine learning, new natural language processing techniques will then be developed to analyze these documents to automatically "learn" the meanings of words and idiosyncratic expressions. New methods will then be developed to automatically discover how the meanings of words and expressions have changed. This research will have a particular focus on social media, because it contains an abundance of new words and usages.******This research will enable the construction of higher quality dictionaries. This will have broad benefits for language technology systems, because they often rely on dictionaries. It will also benefit traditional dictionary users, including language learners and translators.**

语言技术是许多日常软件系统的重要组成部分，例如智能手机上的预测文本输入，文字处理器中的拼写检查器和自动翻译工具。许多语言技术系统依赖于类似词典的资源，这些资源包含有关单词的专门信息。******语言总是在变化。例如，“男人说教”（mansplain）和“宅度假”（staycation）等新词每天都在被创造出来。既有词汇通常会有新的含义，比如与社交媒体相关的“post”和“wall”的新含义。新的表达也出现了，比如“快闪族”。因此，字典必须不断更新以包括这些新用法。******当语言技术系统的字典中缺少信息时，系统将无法正常工作。因此，保持词典的更新对于构建高质量的语言技术至关重要。******字典也是语言学习者和译者的重要工具，在文化遗产方面也很有价值。为了在这些方面发挥最大的作用，字典还必须保持最新。******传统上，词典编纂者手动分析大量文本，以确定有关单词如何使用的相关事实，然后编写字典条目。然而，每天都有惊人数量的文字被写出来，尤其是通过网络和社交媒体等在线资源。词典编纂者根本不可能跟上这一切。******那么，字典怎样才能保持最新呢？******需要使用自动方法。这个研究项目将首先考虑如何最好地从网络上建立大量的文档集合。利用最新的语言学理论和统计机器学习方面的研究，新的自然语言处理技术将被开发出来分析这些文档，以自动“学习”单词和特殊表达的含义。新的方法将被开发出来，自动发现单词和短语的含义是如何变化的。这项研究将特别关注社交媒体，因为它包含了大量的新词和用法。******这项研究将有助于构建更高质量的词典。这将给语言技术系统带来广泛的好处，因为它们经常依赖词典。这也将使传统的词典用户受益，包括语言学习者和翻译