SGER: Exploratory Research: Using the Cuberinfrastructure to build a Full Text Index to the Web

该项目将尝试建立互联网档案馆历史收藏中文字网页的全文索引。自1996年以来，互联网档案馆每两个月就会拍摄一次网络快照，并将其存储起来。它现在包含大约400亿个网页，消耗了数PB的存储空间。该项目的结果索引可能是有史以来创建的最大、组织最好的倒排索引，可供学术研究人员免费使用。它将使社会和信息科学家能够探索当代事件和实践的全新维度，同时为信息科学家提供重要的大规模测试资源，如关于半结构化集合的高级信息检索。

This project will attempt to build a full text index to the textual web pages in the historical collections of the Internet Archive. The Internet Archive has taken a snapshot of the web every two months since 1996 and stored it. It now comprises approximately 40 billion web pages, consuming multiple petabytes of storage.The resulting index of the project may be the largest and best organized inverted index ever created that is freely available to academic researchers. It will enable social and information scientists to explore altogether new dimensions of contemporary events and practices, while offering information scientists a vital large-scale testing resource in areas such as advanced information retrieval on semistructured collections.