Evaluation of text-mining systems for biology: overview of the Second BioCreative community challenge.
Evaluation of text-mining systems for biology: overview of the Second BioCreative community challenge.
复制标题
评估生物学的文本挖掘系统：第二次生物综合社区挑战的概述。
DOI：
10.1186/gb-2008-9-s2-s1
复制
发表时间：
2008
期刊：
影响因子：
12.3
通讯作者：
Valencia A
中科院分区：
文献类型：
--
作者：
Krallinger M;Morgan A;Smith L;Leitner F;Tanabe L;Wilbur J;Hirschman L;Valencia A
基因组科学已经经历了一个有效的文本处理工具，可以从越来越多的出版文献中提取生物相关信息的需求不断增长。为此，最近专门为生物领域开发了一系列文本挖掘和信息提取工具。这些工具只有在被设计为满足现实生活中的任务时才有用，并且它们的性能可以被估计和比较。BioCreative挑战（生物学信息提取的关键评估）包括一个合作倡议，为监测和评估应用于生物学相关问题的最先进的文本挖掘系统提供一个共同的评估框架。第二次生物创新评估（2006年至2007年）吸引了来自全世界13个国家的44个小组，目的是评价目前为本次挑战评估所确定的三项任务中的一项或多项任务开发的信息提取/文本挖掘技术。这些任务包括识别摘要中提到的基因（基因提及任务）;提取摘要中提到的人类基因的唯一标识符列表（基因归一化任务）;最后提取物理蛋白质-蛋白质相互作用注释相关信息（蛋白质-蛋白质相互作用任务）。用于评价第三项任务提交物的“金标准”数据由相互作用数据库MINT（分子相互作用数据库）和IntAct提供。与第一次BioCreative评估相比，第二次BioCreative评估每项任务的参与者数量几乎增加了一倍。对于基因提及的最佳提交物，观察到精确度和召回率平衡的总体改善（F评分0.87）;对于基因标准化任务，与第一次BioCreative挑战中提出的类似任务获得的结果相比，最佳结果相当（F评分0.81）。在蛋白质-蛋白质相互作用任务的情况下，探索了从全文文章中实验确认的注释提取的重要性和困难，根据注释提取工作流程的步骤产生不同的结果。在所有三项任务中观察到的一个共同特点是，系统输出的组合可以产生比任何单一系统更好的结果。最后，第一个文本挖掘元服务器的开发是在这一社区挑战的背景下推动的。
Genome sciences have experienced an increasing demand for efficient text-processing tools that can extract biologically relevant information from the growing amount of published literature. In response, a range of text-mining and information-extraction tools have recently been developed specifically for the biological domain. Such tools are only useful if they are designed to meet real-life tasks and if their performance can be estimated and compared. The BioCreative challenge (Critical Assessment of Information Extraction in Biology) consists of a collaborative initiative to provide a common evaluation framework for monitoring and assessing the state-of-the-art of text-mining systems applied to biologically relevant problems. The Second BioCreative assessment (2006 to 2007) attracted 44 teams from 13 countries worldwide, with the aim of evaluating current information-extraction/text-mining technologies developed for one or more of the three tasks defined for this challenge evaluation. These tasks included the recognition of gene mentions in abstracts (gene mention task); the extraction of a list of unique identifiers for human genes mentioned in abstracts (gene normalization task); and finally the extraction of physical protein-protein interaction annotation-relevant information (protein-protein interaction task). The 'gold standard' data used for evaluating submissions for the third task was provided by the interaction databases MINT (Molecular Interaction Database) and IntAct. The Second BioCreative assessment almost doubled the number of participants for each individual task when compared with the first BioCreative assessment. An overall improvement in terms of balanced precision and recall was observed for the best submissions for the gene mention (F score 0.87); for the gene normalization task, the best results were comparable (F score 0.81) compared with results obtained for similar tasks posed at the first BioCreative challenge. In case of the protein-protein interaction task, the importance and difficulties of experimentally confirmed annotation extraction from full-text articles were explored, yielding different results depending on the step of the annotation extraction workflow. A common characteristic observed in all three tasks was that the combination of system outputs could yield better results than any single system. Finally, the development of the first text-mining meta-server was promoted within the context of this community challenge.
登录
查看更多内容
影响因子：
2.9
作者：
Lopez, Gonzalo;Rojas, Ana;Valencia, Alfonso
通讯作者：
Valencia, Alfonso
生物公约概述：生物学信息提取的批判性评估。
影响因子：
3
作者：
Hirschman L;Yeh A;Blaschke C;Valencia A
通讯作者：
Valencia A
影响因子：
46.9
作者：
Noble, William S.
通讯作者：
Noble, William S.
影响因子：
4.3
作者：
Hirschman, L
通讯作者：
Hirschman, L
影响因子：
3
作者：
Yeh A;Morgan A;Colosimo M;Hirschman L
通讯作者：
Hirschman L