Text Reuse and Information Flow

新闻故事或网页可以包含大量重复使用的信息。不同的作者可能会根据相同的来源呈现一个故事或事件的不同版本，而事件的事实可能会在每次呈现时被概括或重述。有时，这样的演示彼此之间几乎没有共同点；在其他时候，一个演示可能是另一个的副本，只需稍作编辑。给出一个感兴趣的话题，足够广泛的档案可以用来识别特定想法或陈述的起源，并检查它们的有效性。这个项目的目标是开发技术来确定相同信息的替代版本，以便重建信息在文件之间的“流动”。该项目涉及调查一系列在句子、段落和文件一级检测重复使用的方法。这项研究使用了一系列语料库，如新闻、网络爬虫和博客，以探索不同情况下的重用和信息流的维度。这项研究及其结果将对工具的设计产生重大影响，这些工具可用于验证和评估来自不同可靠性来源的信息。这样的工具在教育、科学研究和国家安全方面的许多应用都将是有价值的。研究结果将以论文形式发表，可通过项目网站(http://ciir.cs.umass.edu/research/textreuse.html)获取，源代码将通过流行的狐猴工具包(http://www.lemurproject.org/).)分发

News stories or Web pages can contain a great deal of reused information. Different authors may each present different versions of a story or event based on the same sources, and the facts of an event may get recapitulated or restated each time it is presented. Sometimes such presentations have little in common with each other; at other times one may be a copy of the other with minor edits. Given a topic of interest, then, a sufficiently extensive archive could be used to identify when particular ideas or statements originated and to check their validity. The goal of this project is to develop techniques to identify alternative versions of the same information in order to reconstruct how information "flows" between documents.The project involves the investigation of a range of approaches to detecting reuse at the level of sentences, passages and documents. The research is evaluated using a range of corpora, such as news, Web crawls, and blogs, in order to explore the dimensions of reuse and information flow in different situations. The research and its outcomes will have a significant impact on the design of tools that can be used to validate and assess information that comes from sources of differing reliability. Such a tool would be valuable in many applications in education, scientific research, and national security. The results of the research will be published in papers, will be accessible via the project Web site (http://ciir.cs.umass.edu/research/textreuse.html) and source code will be distributed through the popular Lemur toolkit (http://www.lemurproject.org/).