CAREER: Towards Scalable Datacenter Services with Strong Robustness Guarantees

现代服务器中心已经达到数十万台服务器，现代分布式服务可以分布在全球多个甚至数十个服务器中心。 为了可伸缩性、可用性和性能，这些服务越来越多地采用了一种弱化的数据一致性模型。 这种权衡在诸如网络抓取、搜索和内容分发等应用程序中证明是非常成功的。 另一方面，最近的趋势是将越来越多的动态应用程序转移到“云”上，这预示着服务需求的转变-丢失数据或应用操作乱序可能是不可接受的-同时访问数据和服务的广泛需求也是如此。该研究项目重新考虑了构建具有强大鲁棒性保证和大规模的存储和复制系统的挑战。 认识到大规模的复杂系统通常是由较小的组组成的-它们要么提供不同的功能，要么将一些较大的状态空间划分为较小的，更易于管理的部分-它利用组组合方法来解决问题。 首先，该研究为较小的节点组开发了新型协议，以最小的开销提供强大的属性。 其次，它提出了一个协调服务和一套管理算法，自适应地组织这些群体，并把它们组合在一起。 这些算法探讨了动态负载平衡、拓扑控制和安全等问题。 该研究包括大量的系统构建，包括将其核心服务合并到其他几个分布式系统中（例如，一个可伸缩的虚拟世界和一个独立的名称解析服务）。非常丰富和多样的互联网服务运行的互联网中心形成了人们的在线体验的核心今天。 本研究开发的算法和软件系统可以降低开发可扩展、可靠、安全、高效和易于管理的数据中心服务的障碍。通过降低构建强大的大规模应用程序的技术难度-通过为它们提供坚实的基础-这项研究可能会使互联网服务的进一步创新成为可能。

Modern datacenters have reached hundreds of thousands of servers, and modern distributed services can be spread over multiple or even dozens of datacenters worldwide. For scalability, availability, and performance, these services have increasingly embraced a weakened model of data consistency. This trade-off proved highly successful for applications such as web crawling, search, and content distribution. On the other hand, the recent trend to move ever-more dynamic applications to the "cloud" portends a shift in service requirements---in which losing data or applying operations out-of-order may not be acceptable---as does far-flung demand for concurrent access to data and services.To meet these changing needs, this research project reconsiders the challenge of building storage and replicated systems with strong robustness guarantees and at scale. Recognizing that large-scale, complex systems are typically built-up from smaller groups---which either provide different functionality or partition some larger state-space into smaller, more manageable parts---it leverages a group compositional approach to tackle the problem. First, the research develops novel protocols for smaller groups of nodes that offer strong properties at minimal overhead. Second, it proposes a coordination service and a suite of management algorithms that adaptively organizes these groups and composes them together. These algorithms explore problems of dynamic load balancing, topological control, and security. The research includes substantial systems building, including the incorporation of its core services into several other distributed systems (e.g., a scalable virtual world and a istributed name resolution service).The remarkably rich and varied Internet services run out of datacenters form the core of people's online experiences today. The algorithms and software systems developed by this research may lower the barrier to developing datacenter services that are scalable, reliable, secure, efficient, and easy to manage. And by reducing the technical difficulty of building robust, large-scale applications---by providing them with a firm foundation---this research may enable further innovation in Internet services.