CAREER: Rethinking Replication in Highly Available and Reliable Data Stores

该项目针对构建高可靠和可用的分布式存储系统的长期和日益普遍的挑战，这些系统是大规模Internet服务的关键组件。本文旨在从抽象、性能、通信和故障模型等几个重要方面研究和解决当前分布式存储系统设计中的不足。这些不足通常来自分布式系统的传统设计，这些设计没有为今天的互联网规模做好准备。这项研究可能会对工业和社会产生重大影响，因为分布式系统是云计算、无服务器计算和高性能计算等现代计算基础设施的基石。特别是，这项工作将与广泛使用的Microsoft、b谷歌、MongoDB和roach构建的分布式存储系统协作完成。PI正在与他们的部门合作，在云计算、分布式系统、可靠系统和软件工程等一般领域扩大课程提供的多学科课程。PI将把提案中的主题纳入他所教授的课程中。传统上，系统经常使用一种称为状态机复制（SMR）的方法来实现可靠性。SMR通常被视为一个独立的模块，具有明确的边界，例如键值接口，将其与系统的其余部分隔离开来。然而，这在今天的系统中产生了问题。例如，在地理复制系统中使用SMR作为黑盒可能会导致很大的性能损失，因为每个应用程序请求都可能转换为多个SMR操作，并且每个操作至少需要广域往返才能完成。本文从实践中不同的方面（抽象、性能、通信模型和失效模型）总结了现有系统存在的四大障碍，并针对这些障碍提出了四个研究重点：1)使用事务性数据结构来丰富应用程序和数据存储之间的抽象层，2)研究利用多核资源来提高地理复制的性能，3)在比现状更通用的通信模型下研究先进的SMR技术，4)研究容忍新的故障模型，无声数据损坏。拟议的工作将开发一套新颖的变革性技术，包括一个新的复制事务数据结构库，支持构建一般类型的应用程序，其他新的系统设计和实现，提供快速和一致的复制，将被封装在最先进的研究产品中，并将显著提高现代分布式系统的性能和容错能力。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project targets long-lasting and increasingly pervasive challenges in building highly reliable and available distributed storage systems, which are the key components in large-scale Internet services. The proposal aims to study and address insufficiencies in current distributed storage system designs from several important aspects including their abstraction, performance, communication, and failure models. These insufficiencies typically arise from the conventional designs in distributed systems that were not prepared for today’s Internet scale. This research may produce a major impact on industry and society because distributed systems are the cornerstones of modern computing infrastructures such as cloud computing, serverless computing, and high-performance computing. In particular, this work will be done in collaboration with widely used distributed storage systems built in Microsoft, Google, MongoDB, and Cockroach. The PI is working with their department to broaden the course offerings with multidisciplinary courses in the general area of cloud computing, distributed systems, reliable systems, and software engineering. The PI will incorporate the topics in this proposal into the courses he is teaching.Traditionally, systems often use a method called state machine replication (SMR) to achieve reliability. SMR is often treated as an independent module, with a clear boundary such as a key-value interface isolating it from the rest of the system. However, this creates problems in today’s systems. For example, using SMR as a blackbox in geo-replicated systems could incur a great performance penalty because each application request could translate into multiple SMR operations and each operation takes at least a wide-area round-trip to finish. This proposal summarizes four impediments of existing systems from different aspects in practice (abstraction, performance, communication model, and failure model), and proposes four research thrusts targeting these impediments: 1) use transactional data structures to enrich the abstraction layer between applications and the data store, 2) investigate leveraging multicore resources to improve performance for geo-replication, 3) study advanced SMR techniques under a more generalized communication model than the status quo, 4) study tolerating a new failure model, silent data corruption. The proposed work will develop a set of novel, transformative technologies, including a new replicated transactional data structure library that supports building general types of applications, other new system designs and implementations for providing fast and consistent replication that will be encapsulated in state-of-the-art research products and will dramatically improve the performance and fault-tolerance of modern distributed systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.