An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition.
An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition.
复制标题
DOI：
10.1186/s12859-015-0564-6
复制
发表时间：
2015-04-30
影响因子：
3
通讯作者：
Paliouras G
中科院分区：
文献类型：
--
作者：
Tsatsaronis G;Balikas G;Malakasiotis P;Partalas I;Zschunke M;Alvers MR;Weissenborn D;Krithara A;Petridis S;Polychronopoulos D;Almirantis Y;Pavlopoulos J;Baskiotis N;Gallinari P;Artiéres T;Ngomo AC;Heino N;Gaussier E;Barrio-Alvers L;Schroeder M;Androutsopoulos I;Paliouras G
这篇文章概述了第一届BioASQ挑战赛，这是一项大规模生物医学语义索引和问答（QA）竞赛，于2013年3月至9月举行。BioASQ评估系统对大量生物医学科学文章进行语义索引的能力，并通过结合生物医学文章和本体的信息，对给定的自然语言问题返回简洁和用户可理解的答案。2013年BioASQ竞赛包括两个任务，任务1a和任务1b。在任务1a中，参与者被要求用MeSH标题自动标注新的PubMed文档。12个团队参加了任务1a，总共提交了46个系统运行，其中一个团队的表现始终优于NLM使用的MTI索引器，该索引器用于向管理员建议MeSH标题。任务1b使用基准数据集，包含29个开发和282个测试英语问题，以及黄金标准（参考）答案，由来自欧洲各地的生物医学专家团队准备，参与者必须自动生成答案。三个团队参加了任务1b，共运行了11次系统。BioASQ的基础设施，包括基准数据集、评估机制、参与者的结果和基线方法，都是公开的。已经开发了一个公开可用的生物医学语义索引和QA评估基础设施，其中包括基准数据集，可用于评估以下系统：为已发表的文章或英语问题分配MeSH标题；从PubMed Central的本体、相关文章和片段中检索相关RDF三元组；给出“准确”和分段大小的“理想”答案（摘要）。参加2013年BioASQ竞赛的系统的结果是有希望的。在任务1a中，其中一个系统在NLM的MTI索引器上的表现一直更好。在任务1b中，系统在对“理想”答案的人工评估中获得了高分；因此，他们提供了高质量的摘要作为答案。总的来说，BioASQ帮助获得了一个统一的观点，即如何将文本分类、语义索引、文档和通道检索、问题回答和文本摘要等技术结合起来，使生物医学专家能够获得简洁、用户可理解的问题答案，以反映他们的真实信息需求。本文的在线版本（doi:10.1186/s12859-015-0564-6）包含补充材料，可供授权用户使用。
This article provides an overview of the first BioASQ challenge, a competition on large-scale biomedical semantic indexing and question answering (QA), which took place between March and September 2013. BioASQ assesses the ability of systems to semantically index very large numbers of biomedical scientific articles, and to return concise and user-understandable answers to given natural language questions by combining information from biomedical articles and ontologies. The 2013 BioASQ competition comprised two tasks, Task 1a and Task 1b. In Task 1a participants were asked to automatically annotate new PubMed documents with MeSH headings. Twelve teams participated in Task 1a, with a total of 46 system runs submitted, and one of the teams performing consistently better than the MTI indexer used by NLM to suggest MeSH headings to curators. Task 1b used benchmark datasets containing 29 development and 282 test English questions, along with gold standard (reference) answers, prepared by a team of biomedical experts from around Europe and participants had to automatically produce answers. Three teams participated in Task 1b, with 11 system runs. The BioASQ infrastructure, including benchmark datasets, evaluation mechanisms, and the results of the participants and baseline methods, is publicly available. A publicly available evaluation infrastructure for biomedical semantic indexing and QA has been developed, which includes benchmark datasets, and can be used to evaluate systems that: assign MeSH headings to published articles or to English questions; retrieve relevant RDF triples from ontologies, relevant articles and snippets from PubMed Central; produce “exact” and paragraph-sized “ideal” answers (summaries). The results of the systems that participated in the 2013 BioASQ competition are promising. In Task 1a one of the systems performed consistently better from the NLM’s MTI indexer. In Task 1b the systems received high scores in the manual evaluation of the “ideal” answers; hence, they produced high quality summaries as answers. Overall, BioASQ helped obtain a unified view of how techniques from text classification, semantic indexing, document and passage retrieval, question answering, and text summarization can be combined to allow biomedical experts to obtain concise, user-understandable answers to questions reflecting their real information needs. The online version of this article (doi:10.1186/s12859-015-0564-6) contains supplementary material, which is available to authorized users.
登录
查看更多内容
影响因子：
3
作者：
Jimeno-Yepes AJ;Plaza L;Mork JG;Aronson AR;Díaz A
通讯作者：
Díaz A
DOI：
10.1136/amiajnl-2010-000055
发表时间：
2011-09
期刊：
Journal of the American Medical Informatics Association : JAMIA
影响因子：
--
作者：
Huang M;Névéol A;Lu Z
通讯作者：
Lu Z
影响因子：
5.8
作者：
Hettne, Kristina M.;Stierum, Rob H.;Kors, Jan A.
通讯作者：
Kors, Jan A.
影响因子：
4.8
作者：
Silla, Carlos N., Jr.;Freitas, Alex A.
通讯作者：
Freitas, Alex A.
影响因子：
14.9
作者：
Doms A;Schroeder M
通讯作者：
Schroeder M