CSR: Medium: Very Large Scale Consistent DHTs

现代分布式系统越来越依赖于分布式存储和查找服务。 然而，现有的存储和查找解决方案仅提供高可用性、高可伸缩性和强一致性语义的子集。在这个项目中，我们开发和谐，分布式哈希表（DHT），旨在提供非常大规模的应用程序具有高度可扩展性，一致性和可用的存储和查找。我们在实现这一目标的过程中应对许多重大挑战。首先，互联网具有不可预测的节点和通信故障，在这种情况下保持一致性是一项艰巨的任务。 为了解决这个问题，我们提供了一个抽象的自我管理组的集合，协调，以确保原子更新到分布式状态。 其次，一致性通常以降低可用性为代价。 在我们的系统中，一致性是不可侵犯的安全属性;可用性是通过复制提供的。 第三，一致复制和原子更新的协调机制通常会导致性能损失。 在我们的工作中，一个关键的见解是，一个可以提高性能与协调机制的授权和自主执行。 最后，通过隔离保持组内一致性所需的大部分通信，我们既简化了实现，又提高了可扩展性。 此外，适应不断变化的工作负载和资源约束更容易，因为它可以在分布式状态的一致更新框架内进行。如果成功，由此产生的存储抽象应该会大大简化复杂分布式应用程序的开发。

Modern distributed systems increasingly rely on distributed storage and lookup services. Existing storage and lookup solutions, however, provide only a subset of high availability, high scalability, and strong consistency semantics. In this project, we develop Harmony, a distributed hash table (DHT) aimed at providing very large scale applications with highly scalable, consistent and available storage and lookup. We address many significant challenges in delivering on this goal. First, the Internet has unpredictable node and communication failures, and preserving consistency in this context is a difficult task. To address this issue, we provide the abstraction of a collection of self-managing groups that coordinate to ensure atomic updates to distributed state. Second, consistency often comes at the cost of reduced availability. In our system, consistency is an inviolable safety property; availability is provided through replication. Third, coordination mechanisms for consistent replication and atomic updates often result in performance penalties. A key insight in our work is that one can improve performance with coordination mechanisms for delegation and autonomous execution. Finally, by isolating most of the communication necessary to preserve consistency within a group, we both simplify our implementation and improve its scalability. Further, adaptation to changing workloads and resource constraints is easier because it can take place within a framework for the consistent update of distributed state. If successful, the resulting storage abstraction should greatly simplify the development of complex distributed applications.