STI: Plethora: A Wide-Area Read-Write Object Repository for the Internet

传统的计算基础设施的特点是几个明显的趋势：(I)在过去的十年中，单位成本的存储容量以超过摩尔定律的速度增长；按照这个速度，预计在五年内1TB的IDE存储的成本将不到100美元；(Ii)应用程序通过增加其存储需求来利用存储趋势，它们越来越依赖协作/并发访问，并支持服务/进程移动性；例如，虚拟天文台是一项大规模的合作努力，它将使天文学研究人员能够查询和挖掘可能分布在遥远地理位置的数百兆兆字节的数据，以及(3)基于IP的存储协议的出现，如iSCSI或FCIP，将允许应用程序无缝地访问广域网上的分布式存储。这些趋势推动了基础技术的发展，以构建能够提供分布式存储资源的单一映像的广域读写存储库。这种基础设施的总体设计目标包括客户端性能、全局资源利用率、系统可伸缩性(提供更大资源和用户池的单一逻辑视图)和应用可伸缩性(支持具有大量资源需求的单个应用)。该建议描述了正在进行的研究和建议的工作，以构建用于在广域互联网试验台上实现的半静态对等(P2P)网络的存储库Plthora。与目前许多完全专注于非结构化动态P2P环境的努力不同，Plethora专注于具有强大网络连通性和部分持久网络状态的半静态对等点。在半静态P2P网络中，对等点可能在很长一段时间内(例如计算服务器)保持在网络中的参与者，并且能够提供合理的高可用性和响应时间保证。该存储库集成了局部性增强覆盖网络中的新概念、读写数据的事务语义以及分层版本控制，以及用于健壮性的智能复制。Plethora的核心是一种依赖于两个不同网络的路由机制--一个用于内容缓存的本地覆盖层和一个用于内容定位的全局覆盖层。本地覆盖通过依赖定义自治系统(AS)的现有互联网基础设施来优化网络邻近性。通过这样做，它实现了卓越的对等性能，并优化了全球网络利用率。在此基础上构建的路由核心是对事务语义的支持，它通过在将提交传播到全局覆盖之前在本地覆盖内聚合提交来增强可伸缩性。最后，通过合成需求生成，通过适当的复制将健壮性和可用性结合在一起。该项目旨在全面开发Plethora背后的核心算法和软件，使用模拟(在我们的P2P评估试验床PET上)和实验(使用PlanetLab基础设施)对其进行详尽的验证，并将其部署在选定的应用程序的环境中。

Conventional computing infrastructure is characterized by several unmistakable trends: (i) in the past decade, storage capacity per unit cost has increased at a rate in excess of Moore's law; at this rate, it is predicted that a terabyte of IDE storage will cost less than $100 within five years; (ii) applications have exploited storage trends by increasing their storage requirements, they are increasingly dependent on collaborative/concurrent access, and support service/process mobility; for example, the Virtual Observatory is a large-scale collaborative effort that will allow astronomy researchers to query and mine potentially hundreds of terabytes of data distributed among distant geographical locations, and (iii) the emergence of IP-based storage protocols such as iSCSI or FcIP will allow applications to access distributed storage seamlessly over wide-area networks. These trends motivate the development of foundational technologies for building a wide-area read-write storage repository capable of providing a single image of a distributed storage resource. The overarching design goals of such an infrastructure include client performance, global resource utilization, system scalability (providing a single logical view of larger resource and user pools) and application scalability (enabling single applications with large resource requirements).This proposal describes ongoing research and proposed work towards building such a repository, Plethora, for semi-static peer-to-peer (P2P) networks implemented on a wide-area Internet testbed. In contrast to many current efforts that focus entirely on unstructured dynamic P2P environments, Plethora focuses on semi-static peers with strong network connectivity and a partially persistent network state. In a semistaticP2P network, peers are likely to remain participants in the network over long periods of time (e.g. compute servers), and are capable of providing reasonably high availability and response-time guarantees. The repository integrates novel concepts in locality enhancing overlay networks, transactional semantics for read-write data coupled with hierarchical versioning, intelligent replication for robustness. At the heart of Plethora is a routing mechanism that relies on two distinct networks - a local overlay for content caching and a global overlay for content location. The local overlay optimizes network proximity by relying on existing Internet infrastructure that defines Autonomous Systems (AS). In doing so, it achieves excellent peer performance and optimizes global network utilization. Built upon this routing core is support for transactional semantics, which enhances scalability by aggregating commits within a local overlay before propagating them to the global overlay. Finally, robustness and availability are incorporated by suitable replication through synthetic demand generation. This project aims to fully develop the core algorithms and software underlying Plethora, exhaustively validate it using simulations (on our P2P evaluation testbed, PET)and experimentation (using the PlanetLab infrastructure), and deploy it in the context of selected applications.