Word form similarity computation and application in natural language processing

词形相似度​​计算及其在自然语言处理中的应用

• 批准号: 261284-2007
• 负责人: Kondrak, Grzegorz
• 金额: $ 1.46万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2009
• 资助国家: 加拿大
• 起止时间: 2009-01-01 至 2010-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=426529
• 关键词: Word; form; similarity; computation; application

In natural language processing, words are often treated as abstract entities divorced from their actual form. However, the actual orthographic or phonetic form of words is an important feature that can be utilized in various tasks. The algorithms that I previously developed for computing word similarity have proven to be effective in a variety of applications, including identification of confusable drug names, and matching person names across different scripts. In the current proposal, I describe how I plan to achieve my two principal research goals: continue developing new measures of similarity, and improve the state-of-the-art in several areas involving sequential processing. With respect to the first goal, my specific objectives are: to propose data-driven learning schemes that integrate many-to-many correspondences and sensitivity to context and to design methods that incorporate linguistic knowledge and require no training data. With respect to the second goal, the areas in which I plan to make an impact are: letter-to-phoneme conversion, machine transliteration, statistical machine translation, and cognate identification. The development and application of word similarity methods leads to a number of challenging research problems, and promises to bring a unifying perspective on several distinct tasks that have so far been perceived as unrelated. Based on the experience of the last few years, I am convinced that this line of research will continue to have an impact beyond the areas of its original application.

在自然语言处理中，单词通常被视为与其实际形式分离的抽象实体。然而，单词的实际拼写或语音形式是一个重要的特征，可以在各种任务中使用。我之前开发的用于计算单词相似性的算法已被证明在各种应用中是有效的，包括识别易混淆的药物名称，以及在不同脚本中匹配人名。在目前的提案中，我描述了我计划如何实现我的两个主要研究目标：继续开发新的相似性措施，并提高国家的最先进的几个领域，涉及顺序处理。关于第一个目标，我的具体目标是：提出数据驱动的学习方案，整合多对多的对应关系和对上下文的敏感性，并设计包含语言知识且不需要训练数据的方法。关于第二个目标，我计划影响的领域是：字母到音素转换，机器音译，统计机器翻译和同源识别。词相似度方法的发展和应用导致了一些具有挑战性的研究问题，并有望为迄今为止被认为是无关的几个不同的任务带来统一的观点。根据过去几年的经验，我相信这一研究领域将继续产生超出其最初应用领域的影响。