On lexical resources for digitization of historical documents
On lexical resources for digitization of historical documents
复制标题
历史文献数字化的词汇资源研究
DOI：
--
复制
发表时间：
2009
期刊：
影响因子：
--
通讯作者：
K. Schulz
中科院分区：
文献类型：
--
作者：
Annette Gotscharek;Ulrich Reffle;Christoph Ringlstetter;K. Schulz
许多欧洲图书馆目前正在进行大规模数字化项目，旨在使历史文献和语料库在互联网上在线提供。在这种情况下，适当的词汇资源发挥着双重作用。需要它们来改进历史文档的OCR识别，这目前并没有产生令人满意的结果。其次，即使假设OCR识别完美，由于历史语言与现代语言有很大的不同，提交给搜索引擎的查询与历史文档中搜索词的变体之间的匹配过程需要特殊的支持。虽然专门的词典对这两个问题的有用性似乎是无可争议的，但具体的知识和经验仍然缺乏。对于历史文档的最佳词汇资源应该是什么样的，没有任何提示。优化的词汇资源所带来的真实的好处还不清楚。这两个问题都相当复杂，因为答案取决于文件诞生的历史时间点。我们提出了一系列的实验，阐明这些点。为了进行评估，我们收集了一个涵盖1500年至1950年德国历史文献的大型语料库，并构建了各种类型的词典。我们目前的覆盖率达到了每本字典的十个子时期。额外的实验阐明了OCR准确性和信息检索的改进，可以达到，再次查看不同的字典和时间段。对于OCR和IR，我们的词汇资源导致了实质性的改进。
Many European libraries are currently engaged in mass digitization projects that aim to make historical documents and corpora online available in the Internet. In this context, appropriate lexical resources play a double role. They are needed to improve OCR recognition of historical documents, which currently does not lead to satisfactory results. Second, even assuming a perfect OCR recognition, since historical language differs considerably from modern language, the matching process between queries submitted to search engines and variants of the search terms found in historical documents needs special support. While the usefulness of special dictionaries for both problems seems undisputed, concrete knowledge and experience are still missing. There are no hints about what optimal lexical resources for historical documents should look like. The real benefit reached by optimized lexical resources is unclear. Both questions are rather complex since answers depend on the point in history when documents were born. We present a series of experiments which illuminate these points. For our evaluations we collected a large corpus covering German historical documents from before 1500 to 1950 and constructed various types of dictionaries. We present the coverage reached with each dictionary for ten subperiods of time. Additional experiments illuminate the improvements for OCR accuracy and Information Retrieval that can be reached, again looking at distinct dictionaries and periods of time. For both OCR and IR, our lexical resources lead to substantial improvements.