SBIR Phase I: Incorporation of Knowledge Base into Statistical Machine Translation

这个小企业创新研究（SBIR）第一阶段项目提供了一种创新的机器翻译方法。该项目模型旨在克服开发高质量统计机器翻译（SMT）系统的两个重要瓶颈：(1)无法处理结构问题；(2)对大量平行文本的依赖。当两种语言在结构和词法上存在很大差异时，比如英语和韩语，统计学在处理词序等语法问题上的无能就变得更加明显。该项目是一种学习语言知识的方法，对于自动处理文本中的词序和非局部依赖关系至关重要，并将其与简单的转换一起合并到SMT中，最大限度地发挥基于知识的方法和统计方法的优势，并最大限度地减少对不断增加的双语数据量的需求。该方法旨在构建一个基于句法短语的统计机器翻译引擎，该引擎不仅比现有的基于单词的统计机器翻译引擎更准确，而且能够减少对大型数据源的需求。该项目的主要影响是实现自动翻译质量的潜力，它与最好的基于知识的机器翻译引擎的质量一样高，但同时，它需要最少的手工制作知识，因此在开发时间和人力资源方面的成本要低得多。虽然该研究特别关注英语和韩语之间的机器翻译，但由此产生的翻译模型可能可用于任何语言对之间的翻译。除了直接有利于机器翻译的研究和应用外，该研究还将为从数据中构建双语短语词典提供重大进展，这反过来将有助于跨语言信息检索等多语言任务。Sehda的基于句法短语的机器翻译引擎可以产生无歧义的短语翻译，有助于索引外文文档和构建文档摘要的关键字列表。此外，该项目学习特征以增强传统语言建模的方法将对许多不同的应用产生影响，包括语音识别、搜索引擎、体裁和主题检测以及文档搜索和查询。最后，这项研究通过帮助解决“自动翻译”问题，对美国以及世界其他地区的经济福利和安全至关重要，从而在国内和全球范围内产生有益的影响。

This Small Business Innovation Research (SBIR) Phase I project proffers an innovative approach to machine translation. The project model aims to overcome two important bottlenecks in the development of a high quality Statistical Machine Translation (SMT) system: (1) the inability to handle structural problems, and (2) dependence on huge amounts of parallel texts. The inability of statistics to sufficiently handle grammatical problems such as word order becomes more evident when the language pair is very different in structure and morphology, such as with English and Korean. This project is a method to learn linguistic knowledge crucial to handling word order and nonlocal dependencies automatically from text and incorporate it into SMT along with simple transformations, maximizing the strength of both knowledge-based approaches and statistical approaches, and minimizing the need for ever-increasing amounts of bilingual data. This approach aims to build a syntactic-phrase-based Statistical Machine Translation engine that is not only more accurate than the existing word-based ones but is also capable of decreasing the need for large data sources. The primary impact of the project is the potential for achieving automatic translation quality, which is as high as the quality of the best knowledge-based machine translation engines but which, at the same time, requires a minimum of handcrafting of knowledge and is therefore much lower cost in terms of development time and human resources.While the research is specifically concerned with MT between English and Korean, the resulting translation models would potentially be usable for translation between any pair of languages. In addition to benefiting machine translation research and applications directly, the research will provide significant progress towards building bilingual phrase lexicons from data, which in turn will aid in multi-lingual tasks such as cross-lingual information retrieval. Sehda's syntactic phrase based MT engine can produce unambiguous phrase translations, useful for indexing foreign documents and constructing keyword lists for document summary. Additionally, the project's method to learn features to augment traditional language modeling will have an impact in many different applications including speech recognition, search engines, genre and topic detection, and document search and query. Lastly, this research has beneficial impacts nationally and globally by helping to solve the "automatic translation" problem, an area of paramount importance to the economic welfare and security of the US, as well as to the rest of the world.