权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Collaborative Research: Updating the Militarized Dispute Data Through Crowdsourcing: MID5

协作研究：通过众包更新军事化争端数据：MID5

基本信息

批准号：
1528409
负责人：
Glenn Palmer
金额：
$ 69.04万
依托单位：
Pennsylvania State Univ University Park
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2015
资助国家：
美国
起止时间：
2015-09-15 至 2019-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1528409&HistoricalAwards=false
关键词：
Collaborative Research Updating Militarized Dispute

项目摘要

General Summary The Correlates of War Project's Militarized Interstate Dispute (MID) Data is the most prominent and heavily used data collection in the study of international conflict. The most recent version (MID4) was released in 2014 and brings the period covered to 1816-2010. The MID4 project utilized automated text classification procedures to make the process of identifying relevant news stories more efficient. Over the course of that project, the PIs determined the primary bottleneck in the workflow was the coding of those news documents. To address this inefficiency, The PIs completed a pilot project to determine whether crowdsourcing techniques could be used to code these documents. In the pilot, non-expert workers were paid small sums to read documents and to answer sets of questions, the answers to which were used to identify features of possible militarized incidents (the events that comprise MIDs). A systematic comparison of the crowdsourced responses with those of MID4 Project's trained coders revealed that the crowdsourced codings were completely accurate for 68 percent of the news reports coded; more importantly, high agreement among crowd responses on specific reports was strongly associated with correct coding. This enables the PIs to detect which documents require further expert involvement. As a result, the PIs can produce a majority of the MID data in near-realtime and at limited financial cost. These procedures are applied on the MID5 Project, which will update the MID data for the period 2011-2017.Technical Summary The MID5 project workflow begins with document retrieval from LexisNexis and document classification using the software and methods implemented in MID4. We discard the negatively classified documents, and proceed to extract metadata from the positively classified documents including the document title, the news agency that published the report, the date, and any actors mentioned in the text. Crowd workers are recruited through Amazon's Mechanical Turk and paid a wage to read one of these documents and answer a line of simple, objective questions about it. The questionnaire is predefined, but some extracted metadata is automatically inserted into the questionnaire to improve the quality of responses. Several workers complete a questionnaire for each document, leaving the PIs with problems of aggregation: how to combine multiple worker responses, possibly regarding multiple related questions, into usable data necessary to code the militarized incident. In the pilot study, the PIs show that Bayesian networks are the most effective way to achieve this aggregation. Recently, the PIs have made advances in semi-supervised text classification with hybrid, Deep Restricted Boltzmann Machines, which outperform previous methods in this task.

战争相关者项目的军事化州际争端(MID)数据是国际冲突研究中最突出和使用最频繁的数据收集。最新的版本(MID4)于2014年发布，使所涵盖的时期达到1816-2010年。MID4项目利用自动文本分类程序，使确定相关新闻故事的过程更有效率。在该项目的整个过程中，PIS确定工作流程中的主要瓶颈是这些新闻文档的编码。为了解决这种效率低下的问题，私人投资司完成了一个试点项目，以确定是否可以使用众包技术对这些文件进行编码。在试点中，非专家工作人员获得少量报酬，让他们阅读文件和回答一系列问题，这些问题的答案被用来确定可能发生的军事化事件(构成MID的事件)的特征。对众包反应与MID4项目训练有素的编码员的系统比较表明，众包编码对68%的新闻报道完全准确；更重要的是，群众对特定报道的高度一致与正确的编码密切相关。这使私人投资机构能够检测哪些文件需要进一步的专家参与。因此，PI可以以有限的财务成本近乎实时地产生大部分中期数据。这些程序适用于MID5项目，该项目将更新2011-2017年期间的中期数据。技术摘要MID5项目的工作流程始于从LexisNexis检索文件，并使用MID4中实施的软件和方法对文件进行分类。我们丢弃负面分类的文档，然后从肯定分类的文档中提取元数据，包括文档标题、发布报道的新闻机构、日期以及文本中提到的任何参与者。众包工作人员通过亚马逊的机械土耳其人招募，并支付一定的工资来阅读其中一份文件，并回答一系列简单、客观的问题。调查问卷是预定义的，但一些提取的元数据会自动插入到调查问卷中，以提高回答的质量。几名工作人员为每个文件完成一份调查问卷，给PI留下了汇总的问题：如何将多个工作人员的答复(可能涉及多个相关问题)合并为对军事化事件进行编码所需的可用数据。在试点研究中，PI表明贝叶斯网络是实现这种聚合的最有效方式。最近，PI在混合深度受限Boltzmann机器的半监督文本分类方面取得了进展，在这一任务中表现优于以往的方法。

项目成果

期刊论文数量（7）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Learning a Deep Hybrid Model for Semi-Supervised Text Classification

DOI：
10.18653/v1/d15-1053
发表时间：
2015-09
期刊：
影响因子：
0
作者：
Alexander Ororbia;C. Lee Giles;D. Reitter
通讯作者：
Alexander Ororbia;C. Lee Giles;D. Reitter

Event Ordering with a Generalized Model for Sieve Prediction Ranking

使用筛预测排序的广义模型进行事件排序

DOI：
发表时间：
2017
期刊：
Proceedings of the 8th International Joint Conference on Natural Language Processing
影响因子：
0
作者：
McDowell, William;Chambers, Nathaniel;Ororbia II, Alexander G.;Reitter, David
通讯作者：
Reitter, David

A Framework for Computational Models of Human Memory

人类记忆计算模型的框架