Coding Across the Disciplines: A Project-Based Workshop on Manual Text Annotation Techniques

跨学科编码：基于项目的手动文本注释技术研讨会

• 批准号: 0620673
• 负责人: Stuart Shulman
• 金额: $ 2.5万
• 依托单位: University of Pittsburgh
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-09-01 至 2007-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0620673&HistoricalAwards=false
• 关键词: Coding; Across; Disciplines; Project; Based

This workshop is funded by the Political Science Program and the Studies of Policy, Science and Engineering component of the Science and Society Program. The workshop will bring social and computer scientists together to discuss coding techniques in order to reshape the study of social and political phenomena via the combined use of manual and machine annotation. The longer-term goals are to foster a lasting community of interest focused around the development of new annotation research partnerships and to do so in a way that sets a high scientific standard for consistency, reliability, transparency, and replicability. Coded text corpora are used for basic and applied research in social and computational sciences. Yet the manual annotation of text "the coding" is often conducted in an ad hoc, inconsistent, non-replicable, and unreliable manner. While many researchers from a variety of disciplines stand to benefit from reliably recorded, publicly available, transparent, large-scale observations produced by independent coders, very few current researchers can say with confidence that they know where to acquire, or even how to produce, such annotated text corpora. A primary short-term objective of this workshop is to build on the efforts of a small number of existing projects that feature the manual annotation of text. A related goal is to foster a particular approach to manual coding that is designed to be useful for building algorithms for basic and applied research on social and political issues. The workshop will be small, project-centric, and by invitation only. All of the workshop participants will be together in a seminar-style setting focusing on one project at a time for an extended period. Assuming a maximum of six projects, both days will feature a 2-hour block of time devoted to a single project, as well as a wrap-up session devoted to the collaborative summarization of important workshop findings. All participating projects will be required to prepare and disseminate a specific set of materials in advance of the workshop via a workshop wiki. High quality manual annotation opens up the possibility for cross-disciplinary studies featuring collaboration between social and computational scientists. This second opportunity exists because researchers in the computational sciences, particularly those working in text classification, information retrieval, opinion detection, and variants of natural language processing (NLP), are hungry for the elusive "gold standard" in manual annotation. Reliably coded text corpora of sufficient size and consistency are essential to design and train NLP algorithms.

本次研讨会由政治学计划和科学与社会计划的政策、科学和工程研究部分资助。讲习班将使社会科学家和计算机科学家齐聚一堂，讨论编码技术，以便通过结合使用人工和机器注释来重塑对社会和政治现象的研究。更长期的目标是围绕新的注释研究伙伴关系的发展培养一个持久的利益共同体，并以一种为一致性、可靠性、透明度和可复制性设定高科学标准的方式做到这一点。编码文本语料库用于社会科学和计算科学的基础和应用研究。然而，对文本的手动注释“编码”通常是以特别的、不一致的、不可复制的和不可靠的方式进行的。虽然来自不同学科的许多研究人员将受益于独立编码者产生的可靠记录、公开可用、透明的大规模观察，但目前很少有研究人员能够自信地说，他们知道从哪里获得、甚至如何产生这样的注释文本语料库。该讲习班的一个主要短期目标是在少数现有项目的基础上再接再厉，这些项目的特点是对文本进行手工注释。一个相关的目标是促进一种特别的手工编码方法，这种方法旨在有助于建立关于社会和政治问题的基础和应用研究的算法。研讨会规模较小，以项目为中心，仅限受邀参加。所有的研讨会参与者都将在研讨会式的环境中聚集在一起，在较长的一段时间内一次专注于一个项目。假设最多有六个项目，两天都将有两个小时的时间专门用于单个项目，以及一个总结会议，专门用于协作总结重要的研讨会成果。所有参与的项目将被要求在讲习班之前通过讲习班维基准备和传播一套具体的材料。高质量的手册注释为以社会科学家和计算科学家之间的合作为特色的跨学科研究打开了可能性。第二个机会的存在是因为计算科学中的研究人员，特别是那些从事文本分类、信息检索、观点检测和自然语言处理(NLP)变体的研究人员，渴望在手动注释中找到难以捉摸的“黄金标准”。具有足够大小和一致性的可靠编码文本语料库是设计和训练NLP算法的关键。