Generating finite-state transducers for semi-structured data extraction from the Web
Generating finite-state transducers for semi-structured data extraction from the Web
复制标题
DOI：
10.1016/s0306-4379(98)00027-1
复制
发表时间：
1998-12-01
影响因子：
3.7
通讯作者：
Dung, MT
中科院分区：
文献类型：
--
作者：
Hsu, CN;Dung, MT
整合大量的Web信息源可以显著提高万维网的效用。集成的一个有前途的解决方案是通过使用Web信息中介器，为客户端提供无缝、透明的访问。信息中介需要包装器来访问作为结构化数据库的Web源，但是手工构建包装器是不切实际的。以前的工作对包装器归纳是过于限制，以处理大量的Web页面，包含元组与丢失的属性，多个值，变量属性排列，异常和错别字。本文提出了SoftMealy，一种新的包装器表示形式主义。这种表示是基于有限状态转换器（FST）和上下文规则。这种方法可以包装广泛的半结构化Web页面，因为FST可以编码每个不同的属性排列作为一个路径。SoftMealy包装器可以从少数标记的例子使用我们的泛化算法。我们已经将这种方法实现为一个原型系统，并在真实的Web页面上进行了测试。性能统计数据表明，诱导包装器的大小以及所需的训练工作是线性的测试页面的结构方差。我们的实验还表明，诱导包装器可以推广到看不见的页面。（C）1998 Elsevier Science Ltd.保留所有权利。
Integrating a large number of Web information sources may significantly increase the utility of the World-Wide Web. A promising solution to the integration is through the use of a Web Information mediator that provides seamless, transparent access for the clients. Information mediators need wrappers to access a Web source as a structured database, but building wrappers by hand is impractical. Previous work on wrapper induction is too restrictive to handle a large number of Web pages that contain tuples with missing attributes, multiple values, variant attribute permutations, exceptions and typos. This paper presents SoftMealy, a novel wrapper representation formalism. This representation is based on a finite-state transducer (FST) and contextual rules. This approach can wrap a wide range of semistructured Web pages because FSTs can encode each different attribute permutation as a path. A SoftMealy wrapper can be induced from a handful of labeled examples using our generalization algorithm. We have implemented this approach into a prototype system and tested it on real Web pages. The performance statistics shows that the sizes of the induced wrappers as well as the required training effort are linear with regard to the structural variance of the test pages. Our experiment also shows that the induced wrappers can generalize over unseen pages. (C)1998 Elsevier Science Ltd. All rights reserved.