SBIR Phase I: Customized Metasearch Engine

这个小企业创新研究（SBIR）第一阶段研究项目提出，通过开发一个原型系统来展示公司的元搜索引擎技术的可行性，该原型系统用于基于用户指定的搜索引擎自动创建定制的元搜索引擎（即，搜索引擎的界面页面的URL由用户提供）。元搜索引擎是一种提供对多个现有搜索引擎的统一访问的系统，它是一种有效的机制，可以将多个搜索引擎的覆盖范围联合收割机结合起来，并到达Web上无法公开抓取的部分。必须解决几个问题。首先，需要自动分析包含搜索引擎接口的不同表单标签，以便获得用于连接到搜索引擎的信息（即，提交查询和接收结果）。另一个问题是，由不同的搜索引擎响应于用户查询而返回的各种结果页面需要被自动解析，以便可以正确地提取有用的信息，诸如所检索的文档的URL和片段。第三个问题是，从多个搜索引擎检索到的文档需要自动合并到具有良好检索效率的单个排名列表中，同时最小化下载/分析实际文档的需要。最后一个问题是，创建的元搜索引擎需要自动维护，以便当独立管理的搜索引擎发生变化时，它们可以继续正确工作。该项目的成果将有助于在几个相关领域的研究，如分布式信息检索，自治系统，信息提取和自我维护系统。定制的元数据引擎可以使个人用户和公司/组织在信息获取方面受益。该解决方案节省了用户的时间并提高了工作效率

This Small Business Innovation Research (SBIR) Phase I research project proposes to demonstrate the feasibility of the company's metasearch technology by developing a prototype system for automatically creating customized metasearch engines based on the search engines specified by a user (i.e., the URLs of the interface pages of the search engines are provided by the user). A metasearch engine is a system that provides unified access to multiple existing search engines and it is an effective mechanism to combine the coverage of multiple search engines and to reach the portion of the Web that cannot be publicly crawled. Several issues must be addressed. First, different form tags that contain search engine interfaces need to be automatically analyzed so that information useful for connecting to the search engines (i.e., submitting queries and receiving results) through a program can be correctly extracted. Another issue is that various result pages returned by different search engines in response to user queries need to be automatically parsed so that useful information such as the URLs and snippets of retrieved documents can be correctly extracted. A third issue is that retrieved documents from multiple search engines need to be automatically merged into a single ranked list with good retrieval effectiveness while minimizing the need to downloading/analyzing the actual documents. One final issue is that created metasearch engines need to be automatically maintained so they can continue to work correctly when independently managed search engines change. The outcome of this project will contribute to research in several related areas such as distributed information retrieval, autonomous systems, information extraction and self-maintainable systems. Customized metasearch engines can benefit individual users and companies/organizations in their information acquisition. This solution saves users' time and increases productivity