RIDIR: Collaborative Research: Analytical tools for text based social data integration

当世界上发生一些事情时，比如自然灾害、选举、抗议或政策变化，许多类型的媒体都会对同一事件进行不同的报道。报纸、社交媒体帖子和政府文件都提供了以不同格式存储的事件的独特版本。因为每个来源都提供了自己的视角，综合这些故事大大提高了我们了解事件和媒体环境动态的能力。然而，社会科学家获得这些无数观点的能力有限，因为很少有工具可以自动将这些观点组合成一个综合分析。该项目将提供丰富的基础设施，用于整合记录同一社会现象的不同来源的文本。这种整合往往揭示了许多潜在的社会动态。该项目将开发一种工具，通过四种主要方法将不同格式的文档与相同或密切相关事件的描述集成在一起。首先，该工具将允许用户按主题对齐文档，同时考虑文档之间的结构和风格差异。其次，该工具将根据共享事件或实体编译不同类型的文档。第三，该工具将允许用户提供的模式来组合半结构化文档。最后，该工具将通过识别和解决来自多个来源的矛盾，促进数据融合。该工具将足够灵活，以适应多种研究目的，允许人类的反馈来协助整合，并通过创建一个共同的资源来促进可重复性，该资源可以成为整个学者社区未来研究的基础。该系统本身将适用于几乎任何一组非结构化文本数据，并将广泛适用于整个社会科学的问题。

When something happens in the world -- such as a natural disaster, an election, a protest, or a policy change -- many types of media record different accounts of the same event. Newspapers, social media posts and government documents all provide unique versions of events stored in different formats. Because each source provides its own perspective, synthesizing these stories vastly increase our ability to learn about both events and the dynamics of the media environment. Yet, social scientists are limited in their capacity to access these myriad perspectives because there are few tools for automatically combining these accounts into one integrated analysis. This project will provide a rich infrastructure for integrating texts from diverse sources documenting the same social phenomenon. Such integration often reveals much about underlying social dynamics.This project will develop a tool to integrate documents with different formats with accounts of the same or closely related events through four main methods. First, the tool will allow users to align documents by topic, while accounting for structural and stylistic differences between documents. Second, the tool will compile different types of documents by a shared event or entity. Third, the tool will allow for user-provided schema to combine semi-structured documents. Last, the tool will facilitate data fusion, by identifying and resolving contradictions from multiple sources. The tool will be sufficiently flexible to fit multiple research purposes, allow for human feedback to assist with integration, and facilitate reproducibility by creating a common resource that can be the basis of future research by a whole community of scholars. The system itself will be applicable to almost any set of unstructured text data and will have broad applicability for questions across the social sciences.