Text Reuse and Information Flow

新闻故事或网页可以包含大量重复使用的信息。不同的作者可能会根据相同的来源呈现故事或事件的不同版本，并且每次呈现时事件的事实可能会被重述或重述。有时候，这些演示文稿彼此之间几乎没有共同之处;有时候，一个演示文稿可能是另一个演示文稿的副本，只进行了少量编辑。如果有一个感兴趣的主题，那么，一个足够广泛的档案可以用来确定特定的想法或陈述是何时产生的，并检查它们的有效性。这个项目的目标是开发技术来识别相同信息的替代版本，以重建信息如何在文档之间“流动”。该项目涉及一系列方法的调查，以检测在句子，段落和文档层面的重用。本研究使用一系列的语料库，如新闻，网络爬虫和博客，以探讨在不同情况下的重用和信息流的维度进行评估。这项研究及其成果将对可用于验证和评估来自不同可靠性来源的信息的工具的设计产生重大影响。这种工具在教育、科学研究和国家安全等许多应用中都很有价值。研究结果将以论文形式发表，可通过项目网站（http：//ciir.cs.umass.edu/research/textreuse.html）查阅，源代码将通过流行的狐猴工具包（http：//www.lemurproject.org/）分发。

News stories or Web pages can contain a great deal of reused information. Different authors may each present different versions of a story or event based on the same sources, and the facts of an event may get recapitulated or restated each time it is presented. Sometimes such presentations have little in common with each other; at other times one may be a copy of the other with minor edits. Given a topic of interest, then, a sufficiently extensive archive could be used to identify when particular ideas or statements originated and to check their validity. The goal of this project is to develop techniques to identify alternative versions of the same information in order to reconstruct how information "flows" between documents.The project involves the investigation of a range of approaches to detecting reuse at the level of sentences, passages and documents. The research is evaluated using a range of corpora, such as news, Web crawls, and blogs, in order to explore the dimensions of reuse and information flow in different situations. The research and its outcomes will have a significant impact on the design of tools that can be used to validate and assess information that comes from sources of differing reliability. Such a tool would be valuable in many applications in education, scientific research, and national security. The results of the research will be published in papers, will be accessible via the project Web site (http://ciir.cs.umass.edu/research/textreuse.html) and source code will be distributed through the popular Lemur toolkit (http://www.lemurproject.org/).