CSR: Medium: Very Large Scale Consistent DHTs

CSR：中：超大规模一致的 DHT

• 批准号: 0963754
• 负责人: Thomas Anderson
• 金额: $ 119.92万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-03-15 至 2016-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0963754&HistoricalAwards=false
• 关键词: CSR; Medium; Very; Large; Scale

Modern distributed systems increasingly rely on distributed storage and lookup services. Existing storage and lookup solutions, however, provide only a subset of high availability, high scalability, and strong consistency semantics. In this project, we develop Harmony, a distributed hash table (DHT) aimed at providing very large scale applications with highly scalable, consistent and available storage and lookup. We address many significant challenges in delivering on this goal. First, the Internet has unpredictable node and communication failures, and preserving consistency in this context is a difficult task. To address this issue, we provide the abstraction of a collection of self-managing groups that coordinate to ensure atomic updates to distributed state. Second, consistency often comes at the cost of reduced availability. In our system, consistency is an inviolable safety property; availability is provided through replication. Third, coordination mechanisms for consistent replication and atomic updates often result in performance penalties. A key insight in our work is that one can improve performance with coordination mechanisms for delegation and autonomous execution. Finally, by isolating most of the communication necessary to preserve consistency within a group, we both simplify our implementation and improve its scalability. Further, adaptation to changing workloads and resource constraints is easier because it can take place within a framework for the consistent update of distributed state. If successful, the resulting storage abstraction should greatly simplify the development of complex distributed applications.

现代分布式系统越来越依赖于分布式存储和查找服务。 然而，现有的存储和查找解决方案仅提供高可用性、高可伸缩性和强一致性语义的子集。在这个项目中，我们开发了Harmony，这是一个分布式哈希表（DHT），旨在为超大规模应用程序提供高度可扩展、一致且可用的存储和查找。我们在实现这一目标的过程中应对许多重大挑战。首先，互联网具有不可预测的节点和通信故障，在这种情况下保持一致性是一项艰巨的任务。 为了解决这个问题，我们提供了一个抽象的自我管理组的集合，协调，以确保原子更新到分布式状态。 其次，一致性通常以降低可用性为代价。 在我们的系统中，一致性是不可侵犯的安全属性;可用性是通过复制提供的。 第三，一致复制和原子更新的协调机制通常会导致性能损失。 在我们的工作中，一个关键的见解是，一个可以提高性能与协调机制的授权和自主执行。 最后，通过隔离保持组内一致性所需的大部分通信，我们既简化了实现又提高了其可扩展性。 此外，适应不断变化的工作负载和资源约束更容易，因为它可以在分布式状态的一致更新框架内进行。如果成功的话，最终的存储抽象将大大简化复杂分布式应用程序的开发。