Corpus refactoring: a feasibility study.
Corpus refactoring: a feasibility study.
复制标题
重构语料库：一项可行性研究。
DOI：
10.1186/1747-5333-2-4
复制
发表时间：
2007-09-13
期刊：
影响因子：
--
通讯作者：
Hunter L
中科院分区：
文献类型：
--
作者：
Johnson HL;Baumgartner WA Jr;Krallinger M;Cohen KB;Hunter L
大多数生物医学语料库都没有在创建它们的实验室之外使用，尽管它们提供的黄金标准评估数据的可用性是生物医学文本挖掘进展的限速因素之一。数据表明，一个主要因素影响使用的语料库以外的家庭实验室是它的分布格式。本文测试的假设，语料库重构-改变语料库的格式，而不改变其语义-是一个可行的目标，即它可以完成一个半自动化的过程，并在一个时间效率的方式。我们使用简单的文本处理方法和有限的人工验证将Protein Design Group语料库转换为两种新格式：WordFreak和嵌入式XML。我们跟踪了自动化步骤花费的总时间和成功率。重构后的语料库可以在BioNLP SourceForge网站http://bionlp.sourceforge.net上下载。花费的总时间刚刚超过三个人周，包括大约102小时的编程时间（其中大部分是一次性开发成本）和20小时的自动输出手动验证。此外，重构任何语料库所需的步骤。我们的结论是，重构公开可用的语料库是一种技术和经济上可行的方法，增加使用的数据已经用于评估生物医学语言处理系统。
Most biomedical corpora have not been used outside of the lab that created them, despite the fact that the availability of the gold-standard evaluation data that they provide is one of the rate-limiting factors for the progress of biomedical text mining. Data suggest that one major factor affecting the use of a corpus outside of its home laboratory is the format in which it is distributed. This paper tests the hypothesis that corpus refactoring – changing the format of a corpus without altering its semantics – is a feasible goal, namely that it can be accomplished with a semi-automatable process and in a time-effcient way. We used simple text processing methods and limited human validation to convert the Protein Design Group corpus into two new formats: WordFreak and embedded XML. We tracked the total time expended and the success rates of the automated steps. The refactored corpus is available for download at the BioNLP SourceForge website http://bionlp.sourceforge.net. The total time expended was just over three person-weeks, consisting of about 102 hours of programming time (much of which is one-time development cost) and 20 hours of manual validation of automatic outputs. Additionally, the steps required to refactor any corpus are presented. We conclude that refactoring of publicly available corpora is a technically and economically feasible method for increasing the usage of data already available for evaluating biomedical language processing systems.