Adaptive text correction with Web-crawled domain-dependent dictionaries
Adaptive text correction with Web-crawled domain-dependent dictionaries
复制标题
使用网络爬行的域相关字典进行自适应文本校正
DOI：
--
复制
发表时间：
2007
期刊：
影响因子：
--
通讯作者：
S. Mihov
中科院分区：
文献类型：
--
作者：
Christoph Ringlstetter;K. Schulz;S. Mihov
对于词汇文本校正的成功，对底层背景词典的高覆盖率是至关重要的。尽管如此，大多数更正工具都是建立在静态词典的基础上的，静态词典代表了给定语言的固定表达集合。当处理来自特定领域和领域的文本时，往往会遗漏很大一部分词汇。在这种情况下，自动校正系统和交互校正系统都会产生不太理想的结果。在本文中，我们描述了爬行符合给定输入文本的主题领域的Web页面的策略。引入了特殊的过滤技术，以避免出现许多拼写错误的页面。收集满足输入文本词汇的过滤页面的词汇，获得达到极佳覆盖值的中等大小的动态词典。已经开发出一种工具，可以按照指定的方式自动抓取词典。我们对爬行词典的校正实验表明，使用这些词典，即使是高精度文本的错误率也可以使用完全自动的校正方法来降低。爬行词典处理来自各种主题领域的英语和德语文档集。对于交互式文本纠错，获得了更合理的纠错词候选集，大大减少了人工工作量。为了完成这幅图，我们研究了使用单词三元模型进行校正时的效果。同样，来自爬行语料库的三元组模型的性能优于从静态语料库获得的模型。
For the success of lexical text correction, high coverage of the underlying background dictionary is crucial. Still, most correction tools are built on top of static dictionaries that represent fixed collections of expressions of a given language. When treating texts from specific domains and areas, often a significant part of the vocabulary is missed. In this situation, both automated and interactive correction systems produce suboptimal results. In this article, we describe strategies for crawling Web pages that fit the thematic domain of the given input text. Special filtering techniques are introduced to avoid pages with many orthographic errors. Collecting the vocabulary of filtered pages that meet the vocabulary of the input text, dynamic dictionaries of modest size are obtained that reach excellent coverage values. A tool has been developed that automatically crawls dictionaries in the indicated way. Our correction experiments with crawled dictionaries, which address English and German document collections from a variety of thematic fields, show that with these dictionaries even the error rate of highly accurate texts can be reduced, using completely automated correction methods. For interactive text correction, more sensible candidate sets for correcting erroneous words are obtained and the manual effort is reduced in a significant way. To complete this picture, we study the effect when using word trigram models for correction. Again, trigram models from crawled corpora outperform those obtained from static corpora.