ITR: TeraScale Retrieval

ITR-0082655Newby，Gregory B.北卡罗来纳大学信息与图书馆学学院位于教堂山ITR/SW：Terascale RetrivalTera Scale Retrival致力于大规模信息检索(IR)的科学研究。Terascale检索将通过实施一个用于IR研究和开发的软件工具包，促进实验，以促进信息检索，特别是文本检索的知识。信息检索系统寻求识别满足人类信息需求的文档或文档中的段落。文本检索针对诸如HTML文档等相对非结构化文档以及结构化程度较高的文档(例如，XML)的集合。为了提高信息检索系统的科学知识和性能，需要一种能够快速、实用地实现实验信息检索系统的软件工具包。TeraScale检索工具包将强调对Terscalale数据集的大规模性能：数亿个文档具有TB级的原始数据、数百万个唯一术语、多种语言，以及可能存在数万亿(PB级)的子文档或文档片段。该工具包将强调软件重用、高性能算法以及用于快速原型制作和评估的模块化。TeraScale检索不会迅速从学术用途转向商业化，而是将重点放在实验和评估上，以促进有关信息搜索和使用的科学知识。

ITR-0082655Newby, Gregory B.School of Information and Library Science, Universityof North Carolina at Chapel HillITR/SW: TeraScale RetrievalTeraScale Retrieval addresses the scientific investigation oflarge-scale information retrieval (IR). TeraScale Retrieval willfacilitate experimentation to advance knowledge of informationretrieval, especially text retrieval, by implementing a softwaretoolkit for IR research and development. IR systems seek to identifydocuments or passages from documents that satisfy a human informationneed. Text retrieval is directed at collections of relativelyunstructured documents, such as HTML documents, as well as morestructured documents (e.g., XML). In order to advance scientificknowledge and improve performance of IR, there is a need for asoftware toolkit that enables rapid and practical implementation ofexperimental IR systems. The TeraScale Retrieval toolkit willemphasize large-scale performance with terascale datasets: hundreds ofmillions of documents with terabytes of raw data, millions of uniqueterms, multiple languages, and potential for quadrillions (petascale)of sub-documents or document fragments. The toolkit will emphasizesoftware reuse, high-performance algorithms, and modularity for rapidprototyping and evaluation. Rather than moving quickly from academicuse to commercialization, TeraScale Retrieval will focus onexperimentation and evaluation to contribute to scientific knowledgeabout information seeking and use.