ITR: Data on the Deep Web: Queries, Trawls, Policies and Countermeasures

万维网上超文本的数量与联网数据库中提供的数据量相比相形见绌，据估计，联网数据库的数据量比万维网超文本大400至550倍。 该提案将这些数据称为互联网上的联邦事实和数字，或简称为FFF。这项工作的目标是探索积极利用这一资源的机制和后果。 首先，它描述了利用互联网上的事实和数字的算法和系统。 特别是，它提出了自适应查询处理的电报系统，以适应互联网的波动特性。 它还建议通过在多个数据源上运行递归查询，将Telegraph扩展到从FFF中“拖网”大量数据。 该提案的第二个目标是探讨向广大互联网用户群提供FFF工具的后果。 这包括对政策-社会和技术政策-以及围绕这些政策的经济激励和影响的调查。 第三个目标是探索可以防止FFF技术被滥用的对策的设计空间。

The volume of hypertext on the World Wide Web is dwarfed by the amountof data made available in networked databases, which has been estimated to be 400 to 550 times larger than the WWW hypertext. This proposal refers to this data as the Federated Facts and Figures on the Internet, or simply the FFF. The goal of this work is to explore the mechanisms for -- and consequences of -- aggressively leveraging this resource.The proposal has three aspects. First, it describes algorithms and systems for exploiting facts and figures on the Internet. In particular, it proposes adaptive query processing for the Telegraph system, to adjust to the volatility characteristic of the Internet. It also proposes extending Telegraph to ``trawl'' large amounts of data from the FFF, by running recursive queries over multiple data sources. The second goal of the proposal is to explore the ramifications of providing FFF tools to the broad Internet user base. This includes an investigation of policy -- both social and technical -- and the economic incentives and ramifications surrounding such policies. The third goal is to explore the design space of countermeasures that can prevent FFF technologies from being misused..