Word form similarity computation and application in natural language processing

在自然语言处理中，单词通常被视为与其实际形式分离的抽象实体。然而，单词的实际拼写或语音形式是一个可以在各种任务中利用的重要特征。我之前开发的用于计算单词相似度的算法已被证明在各种应用中都有效，包括识别易混淆的药物名称以及匹配不同文字中的人名。在当前的提案中，我描述了我计划如何实现我的两个主要研究目标：继续开发新的相似性度量，并提高涉及顺序处理的几个领域的最新技术。关于第一个目标，我的具体目标是：提出集成多对多对应性和上下文敏感性的数据驱动学习方案，并设计结合语言知识且不需要训练数据的方法。关于第二个目标，我计划产生影响的领域是：字母到音素的转换、机器音译、统计机器翻译和同源识别。单词相似性方法的开发和应用导致了许多具有挑战性的研究问题，并有望为迄今为止被认为不相关的几个不同任务带来统一的视角。根据过去几年的经验，我相信这一领域的研究将继续产生超出其原始应用领域的影响。

In natural language processing, words are often treated as abstract entities divorced from their actual form. However, the actual orthographic or phonetic form of words is an important feature that can be utilized in various tasks. The algorithms that I previously developed for computing word similarity have proven to be effective in a variety of applications, including identification of confusable drug names, and matching person names across different scripts. In the current proposal, I describe how I plan to achieve my two principal research goals: continue developing new measures of similarity, and improve the state-of-the-art in several areas involving sequential processing. With respect to the first goal, my specific objectives are: to propose data-driven learning schemes that integrate many-to-many correspondences and sensitivity to context and to design methods that incorporate linguistic knowledge and require no training data. With respect to the second goal, the areas in which I plan to make an impact are: letter-to-phoneme conversion, machine transliteration, statistical machine translation, and cognate identification. The development and application of word similarity methods leads to a number of challenging research problems, and promises to bring a unifying perspective on several distinct tasks that have so far been perceived as unrelated. Based on the experience of the last few years, I am convinced that this line of research will continue to have an impact beyond the areas of its original application.