Combining joint models for biomedical event extraction.
Combining joint models for biomedical event extraction.
复制标题
DOI：
10.1186/1471-2105-13-s11-s9
复制
发表时间：
2012-06-26
影响因子：
3
通讯作者：
Manning CD
中科院分区：
文献类型：
--
作者：
McClosky D;Riedel S;Surdeanu M;McCallum A;Manning CD
我们探索了在UMASS和斯坦福生物医学事件提取系统之间执行模型组合的技术。这两个子组件都将事件提取作为结构化预测问题来处理，并使用对偶分解(UMASS)和解析算法(Stanford)来找到最佳评分事件结构。我们的主要关注点是堆叠，在那里，来自斯坦福系统的预测被用作UMassus系统的特征。为了进行比较，我们看看更简单的模型组合技术，如交集和并集，它们只需要每个系统的输出，并直接组合它们。首先，我们发现堆叠显著提高了性能，而交集和并集没有提供显著的好处。其次，我们研究了事件结构的图属性及其对系统组合的影响。最后，我们追踪堆叠模型提出的事件的起源，以确定每个系统在输出的不同组成部分中所扮演的角色。我们了解到，虽然堆叠可以提出在这两个基本模型中都看不到的新事件结构，但这些事件的精度极低。除去这些新事件，我们已经是最先进的F1在Genia的测试集上提高到56.6%(任务1)。总体而言，通过堆叠形成的组合系统(“Faust”)在BioNLP 2011共享任务中表现良好。FAUST系统在四个任务中有三个获得了第1名：Genia任务1(56.0%F1)和任务2(53.9%)的第1名，表观遗传学和翻译后修饰轨道的第2名(35.0%)，传染病轨道的第1名(55.6%)。我们提出了一种最先进的事件提取系统，它依靠结构化预测和通过堆叠的模型组合的优势。与其他任务的结果类似，堆叠的性能优于交集和并集，并产生非常强的结果。模型组合的效用取决于数据的互补视图，我们证明了我们的子系统捕获了不同的事件结构的图形属性。最后，通过剔除低精度的新奇事件，我们证明了堆叠可以进一步提高性能。
We explore techniques for performing model combination between the UMass and Stanford biomedical event extraction systems. Both sub-components address event extraction as a structured prediction problem, and use dual decomposition (UMass) and parsing algorithms (Stanford) to find the best scoring event structure. Our primary focus is on stacking where the predictions from the Stanford system are used as features in the UMass system. For comparison, we look at simpler model combination techniques such as intersection and union which require only the outputs from each system and combine them directly. First, we find that stacking substantially improves performance while intersection and union provide no significant benefits. Second, we investigate the graph properties of event structures and their impact on the combination of our systems. Finally, we trace the origins of events proposed by the stacked model to determine the role each system plays in different components of the output. We learn that, while stacking can propose novel event structures not seen in either base model, these events have extremely low precision. Removing these novel events improves our already state-of-the-art F1 to 56.6% on the test set of Genia (Task 1). Overall, the combined system formed via stacking ("FAUST") performed well in the BioNLP 2011 shared task. The FAUST system obtained 1st place in three out of four tasks: 1st place in Genia Task 1 (56.0% F1) and Task 2 (53.9%), 2nd place in the Epigenetics and Post-translational Modifications track (35.0%), and 1st place in the Infectious Diseases track (55.6%). We present a state-of-the-art event extraction system that relies on the strengths of structured prediction and model combination through stacking. Akin to results on other tasks, stacking outperforms intersection and union and leads to very strong results. The utility of model combination hinges on complementary views of the data, and we show that our sub-systems capture different graph properties of event structures. Finally, by removing low precision novel events, we show that performance from stacking can be further improved.