Word form similarity computation and application in natural language processing

在自然语言处理中，单词通常被视为脱离其实际形式的抽象实体。然而，单词的实际正字法或语音形式是一个重要的特征，可以在各种任务中使用。我之前开发的用于计算单词相似度的算法已被证明在各种应用程序中是有效的，包括识别易混淆的药物名称，以及跨不同脚本匹配人名。在当前的提案中，我描述了我计划如何实现我的两个主要研究目标：继续开发新的相似性测量方法，并在涉及顺序处理的几个领域提高最新水平。关于第一个目标，我的具体目标是：提出数据驱动的学习方案，该方案集成了多对多对应关系和对上下文的敏感性，并设计纳入语言知识且不需要训练数据的方法。关于第二个目标，我打算发挥影响的领域是：字母到音素转换、机器音译、统计机器翻译和同源识别。单词相似度方法的发展和应用导致了许多具有挑战性的研究问题，并有望为迄今为止被认为不相关的几个不同任务带来统一的视角。根据过去几年的经验，我相信这条研究路线将继续在其原始应用领域之外产生影响。

In natural language processing, words are often treated as abstract entities divorced from their actual form. However, the actual orthographic or phonetic form of words is an important feature that can be utilized in various tasks. The algorithms that I previously developed for computing word similarity have proven to be effective in a variety of applications, including identification of confusable drug names, and matching person names across different scripts. In the current proposal, I describe how I plan to achieve my two principal research goals: continue developing new measures of similarity, and improve the state-of-the-art in several areas involving sequential processing. With respect to the first goal, my specific objectives are: to propose data-driven learning schemes that integrate many-to-many correspondences and sensitivity to context and to design methods that incorporate linguistic knowledge and require no training data. With respect to the second goal, the areas in which I plan to make an impact are: letter-to-phoneme conversion, machine transliteration, statistical machine translation, and cognate identification. The development and application of word similarity methods leads to a number of challenging research problems, and promises to bring a unifying perspective on several distinct tasks that have so far been perceived as unrelated. Based on the experience of the last few years, I am convinced that this line of research will continue to have an impact beyond the areas of its original application.