SBIR Phase I: Customized Metasearch Engine

这个小型企业创新研究(SBIR)第一阶段研究项目建议通过开发一个原型系统来证明该公司元搜索技术的可行性，该原型系统基于用户指定的搜索引擎(即，搜索引擎的界面页面的URL由用户提供)自动创建定制的元搜索引擎。元搜索引擎是一种提供对现有多个搜索引擎的统一访问的系统，它是将多个搜索引擎的覆盖范围结合在一起并到达Web上无法公开爬行的部分的有效机制。必须解决几个问题。首先，需要自动分析包含搜索引擎界面的不同表单标签，以便能够正确提取对通过程序连接到搜索引擎(即，提交查询和接收结果)有用的信息。另一个问题是，需要自动解析不同搜索引擎响应用户查询返回的各种结果页面，以便能够正确地提取有用的信息，如URL和检索到的文档的片段。第三个问题是，从多个搜索引擎检索的文档需要自动合并到一个具有良好检索效果的单一排序列表中，同时最大限度地减少下载/分析实际文档的需要。最后一个问题是，创建的元搜索引擎需要自动维护，这样当独立管理的搜索引擎发生变化时，它们才能继续正常工作。该项目的成果将有助于几个相关领域的研究，如分布式信息检索、自主系统、信息提取和自我维护系统。定制的元搜索引擎可以使个人用户和公司/组织在他们的信息获取中受益。该解决方案节省了用户的时间并提高了工作效率

This Small Business Innovation Research (SBIR) Phase I research project proposes to demonstrate the feasibility of the company's metasearch technology by developing a prototype system for automatically creating customized metasearch engines based on the search engines specified by a user (i.e., the URLs of the interface pages of the search engines are provided by the user). A metasearch engine is a system that provides unified access to multiple existing search engines and it is an effective mechanism to combine the coverage of multiple search engines and to reach the portion of the Web that cannot be publicly crawled. Several issues must be addressed. First, different form tags that contain search engine interfaces need to be automatically analyzed so that information useful for connecting to the search engines (i.e., submitting queries and receiving results) through a program can be correctly extracted. Another issue is that various result pages returned by different search engines in response to user queries need to be automatically parsed so that useful information such as the URLs and snippets of retrieved documents can be correctly extracted. A third issue is that retrieved documents from multiple search engines need to be automatically merged into a single ranked list with good retrieval effectiveness while minimizing the need to downloading/analyzing the actual documents. One final issue is that created metasearch engines need to be automatically maintained so they can continue to work correctly when independently managed search engines change. The outcome of this project will contribute to research in several related areas such as distributed information retrieval, autonomous systems, information extraction and self-maintainable systems. Customized metasearch engines can benefit individual users and companies/organizations in their information acquisition. This solution saves users' time and increases productivity