DC: Small: An Integrated Architecture for Federated Search

该项目正在为两个新的联合搜索问题开发一个通用解决方案。通过将海量Web数据集划分为面向主题的“分片”，然后在每个查询中只搜索几个分片，使得海量Web数据集更易于管理，从而显著降低了计算成本。通过使用多种技术来表征每种资源的内容并跟踪其内容和查询流量如何随时间变化的框架，可以更容易地将集中的“垂直搜索服务”集成到通用搜索门户中。将资源定义策略和各种信息引入联合搜索需要解决各种新的资源表示、资源选择和结果合并问题。这项研究还解决了动态资源的需求，并超越了普通的案例分析，以表征联合搜索服务所经历的准确性范围。减少搜索海量网络语料库的计算成本可以对大型网络数据集进行更多的学术研究，并降低网络搜索公司的成本。在门户网站中集成专门的信息服务的综合框架使商业网站搜索门户网站更容易部署新的搜索服务。新的算法正在开放源码的Lemur工具包中传播，从而使其非常容易获得。数据集正在以一种形式发布，使其他研究人员能够精确或密切地重建它们。正在发布问题和相关性判断，以便供其他研究人员使用。这个项目是IIS-0841275中所做研究的扩展：网络搜索引擎的多层索引。项目网址：http://www.cs.cmu.edu/~callan/Projects/IIS-0916553/

This project is developing a general solution to two new federated search problems. Massive web datasets are made more manageable by dividing them into topic-oriented "shards," and then searching only a few shards per query, thus reducing computational costs dramatically. Integration of focused "vertical search services" into general-purpose search portals is made more manageable by a framework that uses multiple techniques to characterize the contents of each resource and track how its content and query traffic change over time. Introducing resource definition policies and diverse information into federated search requires solving a variety of new resource representation, resource selection, and result merging problems. This research is also addressing the requirements of dynamic resources, and looks beyond average case analysis to characterize the range of accuracy that a federated search service experiences. Reducing the computational costs of searching massive web corpora enables greater academic study of large web datasets, and lowers costs for web search companies. A comprehensive framework for integrating specialized information services in web portals makes it easier for commercial web search portals to deploy new search services.New algorithms are being disseminated in the open-source Lemur Toolkit, thus making it very accessible. Datasets are being published in a form that enables them to be recreated precisely or closely by other researchers. Queries and relevance judgments are being published so that they may be used by other researchers. This project is an extension of research done in IIS-0841275, SGER: Multi-Tier Indexing for Web Search Engines.Project URL: http://www.cs.cmu.edu/~callan/Projects/IIS-0916553/