STI: Plethora: A Wide-Area Read-Write Object Repository for the Internet

传统计算基础设施有以下几个明显的趋势：(i)在过去十年中，单位成本的存储容量以超过摩尔定律的速度增长；按照这个速度，预计在五年内，1tb的IDE存储成本将低于100美元；（ii）应用程序通过增加存储需求来利用存储趋势，它们越来越依赖于协作/并发访问，并支持服务/流程的移动性；例如，虚拟天文台是一个大规模的协作项目，它将允许天文学研究人员查询和挖掘分布在遥远地理位置上的潜在数百tb的数据，（iii）基于ip的存储协议（如iSCSI或FcIP）的出现将允许应用程序在广域网上无缝访问分布式存储。这些趋势推动了基础技术的发展，这些技术用于构建能够提供分布式存储资源的单个映像的广域读写存储库。这种基础设施的总体设计目标包括客户端性能、全局资源利用率、系统可伸缩性（提供更大资源和用户池的单一逻辑视图）和应用程序可伸缩性（支持具有大资源需求的单个应用程序）。本提案描述了正在进行的研究和为在广域互联网测试平台上实现的半静态点对点（P2P）网络构建这样一个存储库的建议工作。与目前许多完全关注非结构化动态P2P环境的努力相反，过多关注具有强网络连接性和部分持久网络状态的半静态对等点。在半静态p2p网络中，对等节点可能在很长一段时间内仍然是网络的参与者（例如计算服务器），并且能够提供合理的高可用性和响应时间保证。该存储库集成了一些新概念，包括局部性增强的覆盖网络、读写数据的事务性语义以及分层版本控制、增强鲁棒性的智能复制。overgo的核心是一种路由机制，它依赖于两个不同的网络——用于内容缓存的本地覆盖和用于内容定位的全局覆盖。本地覆盖通过依赖定义自治系统（AS）的现有互联网基础设施来优化网络接近性。这样，它实现了卓越的对等性能，并优化了全球网络利用率。建立在这个路由核心之上的是对事务语义的支持，它通过在将提交传播到全局覆盖之前在本地覆盖内聚合提交来增强可伸缩性。最后，通过合成需求生成的适当复制，将鲁棒性和可用性结合起来。该项目旨在全面开发核心算法和软件，使用模拟（在我们的P2P评估测试平台，PET上）和实验（使用PlanetLab基础设施）对其进行彻底验证，并将其部署在选定的应用程序中。

Conventional computing infrastructure is characterized by several unmistakable trends: (i) in the past decade, storage capacity per unit cost has increased at a rate in excess of Moore's law; at this rate, it is predicted that a terabyte of IDE storage will cost less than $100 within five years; (ii) applications have exploited storage trends by increasing their storage requirements, they are increasingly dependent on collaborative/concurrent access, and support service/process mobility; for example, the Virtual Observatory is a large-scale collaborative effort that will allow astronomy researchers to query and mine potentially hundreds of terabytes of data distributed among distant geographical locations, and (iii) the emergence of IP-based storage protocols such as iSCSI or FcIP will allow applications to access distributed storage seamlessly over wide-area networks. These trends motivate the development of foundational technologies for building a wide-area read-write storage repository capable of providing a single image of a distributed storage resource. The overarching design goals of such an infrastructure include client performance, global resource utilization, system scalability (providing a single logical view of larger resource and user pools) and application scalability (enabling single applications with large resource requirements).This proposal describes ongoing research and proposed work towards building such a repository, Plethora, for semi-static peer-to-peer (P2P) networks implemented on a wide-area Internet testbed. In contrast to many current efforts that focus entirely on unstructured dynamic P2P environments, Plethora focuses on semi-static peers with strong network connectivity and a partially persistent network state. In a semistaticP2P network, peers are likely to remain participants in the network over long periods of time (e.g. compute servers), and are capable of providing reasonably high availability and response-time guarantees. The repository integrates novel concepts in locality enhancing overlay networks, transactional semantics for read-write data coupled with hierarchical versioning, intelligent replication for robustness. At the heart of Plethora is a routing mechanism that relies on two distinct networks - a local overlay for content caching and a global overlay for content location. The local overlay optimizes network proximity by relying on existing Internet infrastructure that defines Autonomous Systems (AS). In doing so, it achieves excellent peer performance and optimizes global network utilization. Built upon this routing core is support for transactional semantics, which enhances scalability by aggregating commits within a local overlay before propagating them to the global overlay. Finally, robustness and availability are incorporated by suitable replication through synthetic demand generation. This project aims to fully develop the core algorithms and software underlying Plethora, exhaustively validate it using simulations (on our P2P evaluation testbed, PET)and experimentation (using the PlanetLab infrastructure), and deploy it in the context of selected applications.