RI: Small: Collaborative Research: Statistical Learning of Language Universals

RI：小型：协作研究：语言共性的统计学习

• 批准号: 1116676
• 负责人: Benjamin Snyder
• 金额: $ 34万
• 依托单位: University of Wisconsin-Madison
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-08-01 至 2015-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1116676&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; Statistical

As modern technology infrastructure spreads throughout the world, the quantity of electronic text, written in hundreds of different languages, continues to grow in size and diversity. Building effective information retrieval, extraction, and translation systems across this vast array of languages currently requires time-consuming and expensive linguistic annotations for each language. Generic, fully unsupervised, methods are unlikely to provide a language independent solution to this problem.Focusing on part-of-speech prediction, this project undertakes a novel approach, combining elements of supervised and unsupervised learning without assuming any specific knowledge of the target language. Instead of treating individual languages as closed systems, language-independent "universals" are statistically estimated from dozens of languages for which annotated corpora exist, and these learned universals are used to predict the part-of-speech categories of unannotated languages. At the heart of the project is a data-driven exploration of language-independent corpus characteristics that relate cross-lingual linguistic categories to surface statistics of text. These learned patterns are incorporated into expressive structured prediction models using novel approximate learning and inference methods developed by the Principal Investigators of the project.Of the world?s spoken languages, hundreds are at risk of immediate extinction and thousands more are likely to disappear over the coming decades. By facilitating the rapid creation of language-independent linguistic analysis tools, the technology developed under this project has the potential to revolutionize the documentation of endangered languages. In the long-term, this research direction will also help realize the full social benefits of the global technology infrastructure by creating intelligent text processing tools for hundreds of low-resource languages.

随着现代技术基础设施遍布世界各地，以数百种不同语言编写的电子文本的数量在规模和多样性方面继续增长。目前，跨大量语言构建有效的信息检索、提取和翻译系统需要为每种语言进行耗时且昂贵的语言注释。通用的、完全无监督的方法不太可能为这个问题提供独立于语言的解决方案。本项目以词性预测为重点，采用了一种新颖的方法，结合了监督学习和非监督学习的元素，而不需要假设目标语言的任何特定知识。与将单个语言视为封闭系统不同，我们从存在带注释的语料库的数十种语言中统计估计出与语言无关的“共相”，并使用这些学习到的共相来预测未带注释的语言的词性类别。该项目的核心是对与语言无关的语料库特征进行数据驱动的探索，这些特征将跨语言的语言类别与文本的表面统计联系起来。这些学习到的模式被整合到富有表现力的结构化预测模型中，使用由该项目的主要研究人员开发的新颖的近似学习和推理方法。世界的？在美国的口语中，有数百种面临着立即灭绝的危险，还有数千种可能在未来几十年内消失。通过促进独立于语言的语言分析工具的快速创建，在这个项目下开发的技术有可能彻底改变濒危语言的记录。从长远来看，这一研究方向还将通过为数百种低资源语言创建智能文本处理工具，帮助实现全球技术基础设施的全部社会效益。