NeTS: Small: Mashup Content Harvesting for an Open Internet

互联网已经从一个小网络变成了一个规模巨大的基础设施。在世界各地数以百万计的服务器和数十亿的网页上托管的Web上可公开访问的数据的数量正在接近Exabyte的极限。考虑到这些数据的巨大规模，并继续以快速的速度增长，该项目询问是否有可能镜像内容，即将其表示为在线已有的大量数据的函数。因此，一个相关的问题是，整个互联网是否可以被视为一个单一的内容分发网络。也就是说，可以通过从公开可用的Web服务下载镜像内容片段来获取所需的内容（这些内容可能不存在于Internet上的任何一个特定位置）吗？PI建议将mashup内容收集作为旨在实现上述目标的综合方法。在这种方法中，用户拥有PI将开发的能力和方法，通过用Web上公开可用的大量数据表示内容来创建或复制内容。通过使用户能够有效地将新数据镜像为现有数据的函数，实现大规模内容分发变得可行，而无需托管任何新的服务器或对等基础设施。与隐写术方法相反，隐写术通过将信息写入载体介质来显式嵌入信息，所提出的系统有效地将数据镜像到现有的公共可用内容上，而无需任何显式写入。智力优势：建议的研究将解决开发和部署mashup内容收集的关键基本问题。主要的挑战是(i)如何通过可公开访问的网站上的信息嵌入数据和跳跃信息，（ii）如何选择一组核心数据段以最大限度地提高镜像效率，（iii）如何设计和部署可扩展和准确表征mashup内容收集数据载体的方法，（iv）哪些基本的Web载体属性能够实现有效的mashup内容镜像和收集。(v)影响系统性能的网页多样性和连通性的下限是什么；（vi）在封闭的国家级Web环境中如何收集mashup内容；（vii）当内容载体仅限于与特定世界语言相关的网页时，如何收集mashup内容。更广泛的影响：该项目有能力通过促进自由开放的互联网的发展产生重大影响。事实上，免费的，即非商业的Web服务已经为Internet上的众多个人和组织创造了巨大的价值。这些服务的共同点是利用世界各地个人的社区贡献。不幸的是，尽管有热情的协作贡献，现有的免费Web服务并不是真正免费的。例如，维基百科必须筹集大量资金来支持其托管成本。Mashup内容收集为在线社区的维持提供了一种可行的方法，无论它们有多小。这既不可能通过威胁或与商业主流Web服务竞争来实现，也不可能影响其性能。PI计划将mashup内容收集设计为易于使用的浏览器插件并进行传播。

The Internet has transformed from a small network into a gigantic infrastructure of enormous scale. The amount of publicly-accessible data on the Web, hosted at millions of servers and billions of Web pages around the world, is approaching the Exabyte limit. Given the enormous size of this data, which continues to grow at a rapid pace, this project asks if it is possible to mirror content, i.e., represent it as a function of the massive data already available online. Consequently, a related question is if the entire Internet can be treated as a single Content Distribution Network. That is, can one harvest the desired content, which may not exist at any one particular location on the Internet, by downloading pieces of mirrored content from the publicly available Web services?The PI proposes mashup content harvesting as a comprehensive approach that aims to accomplish the above goals. In this approach, users have the ability and means, which the PI will develop, to create or replicate content by representing it in terms of the significant amount of data publicly available on the Web. By enabling users to effectively mirror new data as a function of existing data, it becomes feasible to achieve mass-scale content distribution, yet without hosting any new server- or peer-to-peer infrastructure. Contrary to steganography methods, which explicitly embed information by writing it to a carrier medium, the prosed system effectively mirrors data onto existing publicly-available content without any explicit writing.Intellectual Merit: The proposed research will address fundamental questions that are key to developing and deploying mashup content harvesting. The major challenges are (i) how to embed both data and hopping information via information available on publicly accessible sites, (ii) how to select a set of core data segments to maximize mirroring efficiency, (iii) how to devise and deploy methods for scalable and accurate characterization of mashup content harvesting data carriers, (iv) which fundamental Web carrier properties enable effective mashup content mirroring and harvesting, (v) what are the lower-bounds in terms of the Web page diversity and connectivity that affect system performance, (vi) how mashup content harvesting performs in closed national-level Web environments, and (vii) how mashup content harvesting performs when content carriers are limited to Web pages associated with specific world languages.Broader Impact: This project has the capacity to make a significant impact by facilitating the development of a free and open Internet. Indeed, free, i.e., non-commercial, Web services have created enormous value to numerous individuals and organizations on the Internet. Common for such services is to leverage community contributions from individuals around the world. Unfortunately, despite enthusiastic collaborative contributions, existing free Web services aren't really free. For example, Wikipedia must raise large amounts of money to support its hosting costs. Mashup content harvesting provides a feasible approach for online communities to sustain, no matter how small they may be. This is possible to achieve neither by threatening nor competing with commercial mainstream Web services, nor affecting their performance. The PI plans to design and disseminate mashup content harvesting as easy-to-use browser plug-ins.