RIDIR: Collaborative Research: Analytical tools for text based social data integration

当世界上发生了一些事情时，比如自然灾害、选举、抗议或政策变化，许多类型的媒体都会对同一事件进行不同的报道。报纸、社交媒体帖子和政府文件都提供了以不同格式存储的事件的独特版本。 因为每个来源都提供了自己的视角，综合这些故事大大提高了我们了解事件和媒体环境动态的能力。然而，社会科学家在访问这些无数的观点的能力是有限的，因为几乎没有工具可以自动将这些帐户组合成一个综合分析。该项目将提供一个丰富的基础设施，用于整合来自不同来源的记录同一社会现象的文本。 本项目将开发一种工具，通过四种主要方法将不同格式的文件与相同或密切相关的事件的叙述结合起来。 首先，该工具将允许用户按主题调整文件，同时考虑到文件之间的结构和风格差异。 其次，该工具将按共享事件或实体汇编不同类型的文档。 第三，该工具将允许用户提供的模式来联合收割机组合半结构化文档。 最后，该工具将通过识别和解决来自多个来源的矛盾来促进数据融合。 该工具将具有足够的灵活性，以适应多种研究目的，允许人类反馈以协助整合，并通过创建一个共同的资源来促进可重复性，该资源可以成为整个学者社区未来研究的基础。该系统本身将适用于几乎任何非结构化文本数据集，并将广泛适用于社会科学领域的问题。

When something happens in the world -- such as a natural disaster, an election, a protest, or a policy change -- many types of media record different accounts of the same event. Newspapers, social media posts and government documents all provide unique versions of events stored in different formats. Because each source provides its own perspective, synthesizing these stories vastly increase our ability to learn about both events and the dynamics of the media environment. Yet, social scientists are limited in their capacity to access these myriad perspectives because there are few tools for automatically combining these accounts into one integrated analysis. This project will provide a rich infrastructure for integrating texts from diverse sources documenting the same social phenomenon. Such integration often reveals much about underlying social dynamics.This project will develop a tool to integrate documents with different formats with accounts of the same or closely related events through four main methods. First, the tool will allow users to align documents by topic, while accounting for structural and stylistic differences between documents. Second, the tool will compile different types of documents by a shared event or entity. Third, the tool will allow for user-provided schema to combine semi-structured documents. Last, the tool will facilitate data fusion, by identifying and resolving contradictions from multiple sources. The tool will be sufficiently flexible to fit multiple research purposes, allow for human feedback to assist with integration, and facilitate reproducibility by creating a common resource that can be the basis of future research by a whole community of scholars. The system itself will be applicable to almost any set of unstructured text data and will have broad applicability for questions across the social sciences.