Evaluating Scalable Distributed Erlang for Scalability and Reliability
Evaluating Scalable Distributed Erlang for Scalability and Reliability
复制标题
DOI：
10.1109/tpds.2017.2654246
复制
发表时间：
2017-08
影响因子：
5.3
通讯作者：
Natalia Chechina;K. Mackenzie;S. Thompson;P. Trinder;Olivier Boudeville;Viktoria Fordós;Csaba Hoch;A. Ghaffari;M. Hernandez
中科院分区：
文献类型：
--
作者：
Natalia Chechina;K. Mackenzie;S. Thompson;P. Trinder;Olivier Boudeville;Viktoria Fordós;Csaba Hoch;A. Ghaffari;M. Hernandez
拥有数百台主机和数万个核心的大型服务器正在变得普遍。为了利用这些平台，软件必须既可扩展又可靠，而像 Erlang 这样的分布式参与者语言是该领域经过验证的技术。虽然分布式 Erlang 在概念上支持大规模可靠系统的工程，但实际上它具有一些可扩展性限制，迫使开发人员大规模偏离标准语言机制。在早期的工作中，我们探索了这些可扩展性限制，并通过提供可扩展分布式 (SD) Erlang 库来解决这些限制，该库将 Erlang 虚拟机 (VM) 网络划分为可扩展组 (s_groups)。本文首次对 SD Erlang s_groups 和相关工具进行系统评估，以及如何使用它们。我们使用三个典型的基准测试和案例研究对 SD Erlang 的可扩展性和可靠性进行了全面评估。我们证明 s_groups 提高了最多 256 个主机（6,144 个内核）上可靠和不可靠的 Erlang 应用程序的可扩展性。我们证明 SD Erlang 保留了领先的分布式 Erlang 可靠性模型，但扩展性远远好于标准模型。我们提出了一种新颖的、系统的、工具支持的方法，用于将分布式 Erlang 应用程序重构为 SD Erlang。我们概述了用于大规模 SD Erlang 应用程序的新的和改进的监控、调试和部署工具。我们演示了包含多达 10 K Erlang VM 的系统上关键工具的扩展特性。
Large scale servers with hundreds of hosts and tens of thousands of cores are becoming common. To exploit these platforms software must be both scalable and reliable, and distributed actor languages like Erlang are a proven technology in this area. While distributed Erlang conceptually supports the engineering of large scale reliable systems, in practice it has some scalability limits that force developers to depart from the standard language mechanisms at scale. In earlier work we have explored these scalability limitations, and addressed them by providing a Scalable Distributed (SD) Erlang library that partitions the network of Erlang Virtual Machines (VMs) into scalable groups (s_groups). This paper presents the first systematic evaluation of SD Erlang s_groups and associated tools, and how they can be used. We present a comprehensive evaluation of the scalability and reliability of SD Erlang using three typical benchmarks and a case study. We demonstrate that s_groups improve the scalability of reliable and unreliable Erlang applications on up to 256 hosts (6,144 cores). We show that SD Erlang preserves the class-leading distributed Erlang reliability model, but scales far better than the standard model. We present a novel, systematic, and tool-supported approach for refactoring distributed Erlang applications into SD Erlang. We outline the new and improved monitoring, debugging and deployment tools for large scale SD Erlang applications. We demonstrate the scaling characteristics of key tools on systems comprising up to 10 K Erlang VMs.