SBIR Phase I: Unsupervised Extraction of Relational Data from the Web

这个小型企业创新研究(SBIR)第一阶段研究项目将使软件系统能够利用Web上的数据。语义网旨在允许软件应用程序共享和使用数据。不幸的是，在当今世界，Web上的数据通常不能被大多数应用程序访问，因为它是以人类而不是计算机可用的格式呈现的。最终目标是创建Web上数据的关系视图，以便应用程序可以基于实体及其关系访问Web数据。该项目建议通过一种无监督的机器学习方法来实现这一点，该方法从网站提取数据并将其转换为关系形式。它将开发和实现一个非监督算法，该算法利用网站上发现的多种不同类型的模式，包括链接结构、格式约定和内容规则。该项目将带来强大的新一代网络采集技术，具有明显的商业价值。此外，它还将使语义网的愿景成为现实。Web收集是各种垂直市场日益增长的商业兴趣领域，包括销售情报、市场情报、新闻聚合和背景搜索。然而，今天的网络采集技术是有限的，因为收集丰富、详细的数据必须逐个站点进行。这里描述的方法如果成功，将使新一代智能Web收获技术能够扩展到整个Web。最终，我们的方法将使应用程序能够像查询关系数据库一样查询整个Web。这具有巨大的商业价值，而且还将使许多新类型的Web应用程序得以开发。除了商业价值外，这种技术方法是新颖的，本身就有显著的优点。如果成功，所提出的方法应该推广到其他复杂的领域(如场景理解和自然语言处理)，这些领域必须分析多种不同类型的结构以发现潜在的含义

This Small Business Innovation Research (SBIR) Phase I research project will enable software systems to make use of data on the Web. The semantic web is intended to allow data to be shared and used by software applications. Unfortunately, in the present world, data on the Web is generally inaccessible to most applications because it is presented in a format intended to be usable by humans, as opposed to computers. The ultimate goal is to create a relational view of data on the web, so that applications can access Web data based on entities and their relations. This project proposes to achieve this with an unsupervised machine learning approach that extracts data from web sites and converts it into relational form. It will develop and implement an unsupervised algorithm that takes advantage of multiple heterogeneous types patterns found on web sites, including the link structure, formatting conventions, and content regularities. This project will result in a powerful new generation of Web harvesting technology that has clear commercial value. In addition moreover, it will enable the vision of the semantic web to become a reality. Web harvesting is an area of growing commercial interest for a variety of vertical markets, including Sales Intelligence, Market Intelligence, News Aggregation, and Background Search. However, web-harvesting technology is limited today, since the collection of rich, detailed data must be done on a site-by-site basis. The approach described here, if successful, will enable a new generation of intelligent Web harvesting technology that can scale to the entire Web. Ultimately, our approach will enable applications to query the entire Web as if it were a relational database. This has tremendous commercial value, and moreover, will enable many new types of web applications to be developed. In addition to the commercial value, the technical approach is novel and has significant merits on its own. If it is successful, the proposed method should generalize to other complex domains (such as scene understanding and natural language processing) where multiple heterogeneous types of structure must be analyzed to discover underlying meaning