III: Large: Collaborative Research: Web Archive Cooperative

III：大型：协作研究：网络档案合作社

• 批准号: 1009392
• 负责人: Michael Nelson
• 金额: $ 39.98万
• 依托单位: Old Dominion University Research Foundation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-08-01 至 2014-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1009392&HistoricalAwards=false
• 关键词: III; Large; Collaborative; Research; Web

Web Science is an emerging discipline that studies the Web: how human activity is shaped by Web interactions, how the Web can benefit society, and how Web technologies can be improved. Central to Web Science is access to data that records the history of the Web, as well as data that records human activity (e.g., posed queries, tagged pages, Twitter updates). It is currently very difficult for academic researchers to obtain such Web data because it is hard to locate, it is fragmented across diverse sites, and is recorded using inconsistent formats and strategies. This project will build a Web Archive Cooperative (WAC) that will integrate existing archives (repositories of Web data), making it feasible to access large volumes of data in a simplified fashion. The WAC will be a virtual service, providing search facilities and access mechanisms to existing resources. These resources will not just be Web pages, but all types of available Web information, such as query logs, tag annotations, blogs, profiles and Twitter updates. Furthermore, resources will also include the software tools for building and managing Web archives.The project will explore three goals for a resource discovery service: (1) the manual or automated discovery of entire existing Web related archives; (2) the selection among known archives of the ones that support a specific research question; and (3) the identification of individual resources from within the selected archives. Tools for characterizing discovered archives, especially for the case where the archive does not provide rich descriptive metadata, will also be developed. Characterization of an archive includes elements such as an estimate of the archive's coverage, particulars of the crawling parameters, like dates/frequencies, crawl duration, depth, per-site ceiling on the number of collected pages, content statistics, and link structure. Mechanisms for integrating diverse archives will be developed, and the mechanisms will be applied to site reconstruction (from various archives) and archive views (a logical fusion of resources from multiple sources). Since integration issues are so challenging, an experimental testbed will be set up with small but diverse resources. The testbed will contain several crawls of the same target sites, each obtained with different crawlers and using different parameters. The testbed will also contain related resources. Storage trading schemes will be developed, allowing members to trade local backup space for remote space. A Web archive replication tool will be developed based on existing notions for self-preserving objects. Alternatives for replica synchronization will be studied.Workshops to bring together key Web Science researchers will be organized to discuss available resources and impediments to sharing. These workshops will drive research and identify needed tools and protocols. With small groups of participants, challenge problems will be established, e.g., combining a set of Web archives. Reports of these results at future workshops can incentivize others to participate in the WAC. In addition, an Advisory Board of industrial, government, and academic experts has been set up to guide the project. A Summer Institute for Web Science graduate students will be held. At this Institute, students will learn to use the latest tools and will learn from each other's experiences in dealing with Web data. In addition, a one-day workshop will be developed, to be offered at Web Science conferences (WWW, SIGIR, etc.) to educate participants about WAC resources. An undergraduate Web Sciences track for computer science majors will be set up, taking advantage of WAC resources. The project will have impact in two ways. First, it will provide tools and services that facilitate access to Web resources. Any researcher, from a computer scientist studying efficient Web search, to a social scientist studying how human beliefs are changing today, to a historian studying how the early Web evolved, to a biologist understanding how disease spreads, will benefit from the work. Second, the project motivates students and young researchers to stay in academia. Currently top talent is flowing to industry because only they have comprehensive Web data, and it is so hard to do significant Web Science at universities. The WAC can provide an alternative, attracting more researchers and teachers to this important area.

网络科学是一门新兴的学科，研究网络：网络交互如何塑造人类活动，网络如何造福社会，以及如何改进网络技术。网络科学的核心是访问记录网络历史的数据以及记录人类活动的数据（例如，构成的查询，标记的页面，Twitter更新）。目前学术研究人员很难获得这样的网络数据，因为它很难定位，它分散在不同的网站，并使用不一致的格式和策略记录。该项目将建立一个网络档案合作社，将现有的档案（网络数据储存库）整合在一起，使其能够以简化的方式访问大量数据。WAC将是一种虚拟服务，提供搜索设施和访问现有资源的机制。这些资源将不仅仅是网页，而是所有类型的可用Web信息，如查询日志、标签注释、博客、个人资料和Twitter更新。此外，资源还将包括用于建立和管理网络档案的软件工具，该项目将探讨资源发现服务的三个目标：（1）手动或自动发现整个现有的与网络有关的档案;（2）在已知的档案中选择支持特定研究问题的档案;（3）从选定的档案中识别个别资源。还将开发用于说明已发现档案的特征的工具，特别是在档案没有提供丰富的描述性元数据的情况下。归档的特征包括诸如归档覆盖范围的估计、爬行参数的细节（如日期/频率、爬行持续时间、深度、每个站点收集的页面数量的上限、内容统计和链接结构）等元素。 将建立整合各种档案的机制，并将这些机制应用于网站重建（来自各种档案）和档案视图（来自多种来源的资源的逻辑融合）。由于整合问题是如此具有挑战性，将建立一个实验性的测试平台，使用少量但多样化的资源。测试床将包含相同目标站点的几个抓取，每个抓取使用不同的抓取工具和不同的参数获得。测试平台还将包含相关资源。 将制定存储交易计划，允许成员以本地备份空间换取远程空间。一个网络档案复制工具将开发基于现有的自我保存对象的概念。将研究副本同步的替代方案。将组织研讨会，汇集主要的Web科学研究人员，讨论可用的资源和共享的障碍。这些讲习班将推动研究并确定所需的工具和协议。对于小规模的参与者，将建立挑战问题，例如，结合了一系列的网络档案。在今后的讲习班上报告这些成果可以激励其他人参加妇女咨询委员会。此外，还成立了一个由工业、政府和学术专家组成的咨询委员会，以指导该项目。 将举办网络科学研究生暑期研究所。在这个研究所，学生将学习使用最新的工具，并将相互学习处理Web数据的经验。此外，将开发一个为期一天的研讨会，在网络科学会议（WWW，SIGIR等）上提供。教育参与者了解WAC资源。利用WAC资源，将为计算机科学专业的本科生建立一个网络科学轨道。该项目将在两个方面产生影响。 第一，它将提供便利获取网络资源的工具和服务。任何研究人员，从研究高效网络搜索的计算机科学家，到研究当今人类信仰如何变化的社会科学家，到研究早期网络如何演变的历史学家，再到了解疾病如何传播的生物学家，都将从这项工作中受益。 其次，该项目激励学生和年轻研究人员留在学术界。目前，顶尖人才正流向行业，因为只有他们拥有全面的Web数据，而在大学里很难做重要的Web科学。 WAC可以提供一个替代方案，吸引更多的研究人员和教师到这个重要的领域。