SHINRA: Structuring Wikipedia by Collaborative Contribution
SHINRA: Structuring Wikipedia by Collaborative Contribution
复制标题
SHINRA：通过协作贡献构建维基百科
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Kouta Nakayama
中科院分区：
文献类型：
--
作者：
S. Sekine;Akio Kobayashi;Kouta Nakayama
我们正在报告SHINRA项目，一个用协作构建计划构建维基百科的项目。该项目的目标是创建一个巨大的，结构良好的知识库，用于NLP应用程序，如QA，对话系统和可解释的NLP系统。它是基于“资源合作贡献（RbCC）”计划创建的。我们进行了构建维基百科的共同任务，同时，提交的结果用于构建知识库。目前有CYC、DBpedia、YAGO、Freebase Wikidata等机器可读的知识库，但每一个都有需要解决的问题。CYC有一个覆盖问题，其他人有一个连贯性问题，因为这些都是基于维基百科和/或由许多但本质上不连贯的人群工作者创建的。为了解决后一个问题，我们开始了一个使用自动知识库构建共享任务来构建维基百科的项目。基于共享任务的知识库自动构建技术是一个研究热点。然而，这些任务的目的只是比较不同系统的性能，并找出哪个系统排名最好的有限的测试数据。参与系统的结果不共享，一旦任务结束，系统可能会被放弃。我们认为这种情况可以通过以下改变来改善：1。设计共享任务来构建知识库，而不是只评估有限的测试数据2.使所有系统的输出向公众开放，这样我们就可以运行集成学习，以创造比最好的系统更好的结果。重复任务，以便我们可以运行任务与更大和更好的训练数据从输出的前一个任务（自举和主动学习）我们进行了“SHINRA 2018”与上述方案，并在本文中，我们报告的结果和未来的方向项目。任务是从维基百科页面中提取预定义属性的值。我们将日语维基百科中的大部分实体（即73万个实体）归类为200个ENE类别。基于这些数据，共享任务是从维基百科页面中提取属性的值。我们给出了600个训练数据，要求参与者提交同一类别类型的所有剩余实体的属性值。然后，每个类别中的100个数据用于评估共享任务中的系统输出。我们对输出进行了初步的集成学习，发现在一个类别上有15个F1分数的改进，在我们测试的所有5个类别上，在一个强基线上平均有8个F1分数的改进。基于这一令人鼓舞的结果，我们决定在2019年进行三项任务：多语言分类任务（ML），使用更大的训练数据（JP-5）提取日语中相同的5个类别，以及提取日语中的34个新类别（JP-34）。
We are reporting the SHINRA project, a project for structuring Wikipedia with collaborative construction scheme. The goal of the project is to create a huge and well-structured knowledge base to be used in NLP applications, such as QA, Dialogue systems and explainable NLP systems. It is created based on a scheme of â€Resource by Collaborative Contribution (RbCC)â€. We conducted a shared task of structuring Wikipedia, and at the same, submitted results are used to construct a knowledge base. There are machine readable knowledge bases such as CYC, DBpedia, YAGO, Freebase Wikidata and so on, but each of them has problems to be solved. CYC has a coverage problem, and others have a coherence problem due to the fact that these are based on Wikipedia and/or created by many but inherently incoherent crowd workers. In order to solve the later problem, we started a project for structuring Wikipedia using automatic knowledge base construction shared-task. The automatic knowledge base construction shared-tasks have been popular and well studied for decades. However, these tasks are designed only to compare the performances of different systems, and to find which system ranks the best on limited test data. The results of the participated systems are not shared and the systems may be abandoned once the task is over. We believe this situation can be improved by the following changes: 1. designing the shared-task to construct knowledge base rather than evaluating only limited test data 2. making the outputs of all the systems open to public so that we can run ensemble learning to create the better results than the best systems 3. repeating the task so that we can run the task with the larger and better training data from the output of the previous task (bootstrapping and active learning) We conducted “SHINRA2018†with the above mentioned scheme and in this paper we report the results and the future directions of the project. The task is to extract the values of the pre-defined attributes from Wikipedia pages. We have categorized most of the entities in Japanese Wikipedia (namely 730 thousand entities) into the 200 ENE categories. Based on this data, the shared-task is to extract the values of the attributes from Wikipedia pages. We gave out the 600 training data and the participants are required to submit the attribute-values for all remaining entities of the same category type. Then 100 data out of them for each category are used to evaluate the system output in the shared-task. We conducted a preliminary ensemble learning on the outputs and found 15 F1 score improvement on a category and the average of 8 F1 score improvements on all 5 categories we tested over a strong baseline. Based on this promising results, we decided to conduct three tasks in 2019; multi-lingual categorization task (ML), extraction for the same 5 categories in Japanese with a larger training data (JP-5) and extraction for 34 new categories in Japanese (JP-34).
影响因子：
22.7
作者：
Vrandecic, Denny;Kroetzsch, Markus
通讯作者：
Kroetzsch, Markus