EAGER: Converting Print Dictionaries to Machine-Interpretable Format

EAGER：将印刷词典转换为机器可解释的格式

• 批准号: 1644606
• 负责人: Michael Maxwell
• 金额: $ 7.48万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-15 至 2018-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1644606&HistoricalAwards=false
• 关键词: EAGER; Converting; Print; Dictionaries; Machine

A dictionary documents the building blocks of a language -- its words and idiomatic phrases, with descriptions of their pronunciations, grammatical properties, meanings and uses -- and is an essential component of language documentation, together with a reference grammar and transcribed texts and recordings. Until recently, dictionaries were compiled and organized by hand, entered into some kind of typesetting system, and finally rendered in print form for use by scholars and language learners. Contemporary dictionaries are now compiled and organized electronically so that the information they contain can be used not only to produce stand-alone print artifacts, but also be integrated with the other components to ensure greater accuracy of the documentation as a whole, enable updates to be produced at regular intervals, and support the development of natural-language processing tools for the languages that are documented in this way. The goal of this exploratory project is to develop methods for machines to understand the implicit structure of the hundreds of extant print dictionaries of endangered and other low-resource languages as a critical first step in enabling their documentation to be of maximal usefulness to future generations.Print dictionaries use ordering, typeface and other formatting conventions to indicate the intended structure of dictionary entries. The first task of this project is to use optical character reading (OCR) software to convert those entries to machine-interpretable form so as to preserve the original formatting. The second is to develop software to convert the corrected OCR output into structured, machine-interpretable archive-standard formats. Because print formats vary widely across dictionaries, human intervention is required to inform the software about how to translate the implicit representations for a particular dictionary's entries into explicit ones. But such manual annotation is only required for a small part of the dictionary, as the formatting conventions are consistent across all of its entries, and once learned can be used to identify and correct errors and inconsistencies, and enable automated editing tasks like updating orthographies. The tool will be developed, tested and evaluated using print dictionaries of two indigenous languages of Latin America that were produced in the latter part of the twentieth century. This project is jointly supported by the Documenting Endangered Languages Program in the Behavioral and Cognitive Sciences Division and by the Robust Intelligence Program in the Information and Intelligent Systems Division.

词典记录一种语言的组成部分-其单词和习惯用语，并说明其发音、语法特性、含义和用途-是语言文献的重要组成部分，还有参考语法、转录文本和录音。直到最近，词典都是手工编纂和组织的，输入某种排版系统，最后以印刷形式呈现，供学者和语言学习者使用。当代词典现在都是以电子方式编纂和组织的，因此，它们所包含的信息不仅可以用来制作独立的印刷品，而且还可以与其他组成部分相结合，以确保整个文献的更高准确性，使更新能够定期进行，并支持为以这种方式记录的语言开发自然语言处理工具。这个探索性项目的目标是为机器开发方法，以理解濒危语言和其他低资源语言的数百种现存印刷词典的隐含结构，作为使其文档对后代最大有用性的关键第一步。印刷词典使用排序，字体和其他格式约定来指示词典条目的预期结构。本项目的第一项任务是使用光学字符阅读（OCR）软件将这些条目转换为机器可读形式，以保留原始格式。第二个是开发软件，将校正后的OCR输出转换为结构化的、机器可解释的档案标准格式。由于打印格式在不同的字典中差异很大，因此需要人工干预来通知软件如何将特定字典条目的隐式表示转换为显式表示。但这种手动注释只需要字典的一小部分，因为格式约定在其所有条目中是一致的，并且一旦学习就可以用于识别和纠正错误和不一致，并实现自动编辑任务，如更新拼写。将利用20世纪后期制作的拉丁美洲两种土著语言的印刷词典来开发、测试和评价这一工具。该项目由行为和认知科学司的濒危语言记录计划以及信息和智能系统司的强大智能计划共同支持。

项目成果

Endangered Data for Endangered Languages: Digitizing Print dictionaries

濒危语言的濒危数据：数字化印刷词典

• DOI: 10.18653/v1/w17-0112
• 发表时间: 2017
• 期刊: Proceedings of the 2nd Workshop on the Use of Computational Methods in the Study of Endangered Languages
• 作者: Maxwell, Michael;Bills, Aric
• 通讯作者: Bills, Aric