FileScale: Fast and Elastic Metadata Management for Distributed File Systems
FileScale: Fast and Elastic Metadata Management for Distributed File Systems
复制标题
DOI：
10.1145/3620678.3624784
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
Gangjun Liao;Daniel J. Abadi
中科院分区：
文献类型：
--
作者：
Gangjun Liao;Daniel J. Abadi
在单个机器上或通过共享磁盘抽象存储元数据的文件系统面临着可伸缩性挑战，特别是在需要管理数十亿文件的环境中。最近的工作表明，采用无共享的分布式数据库系统（DDBMS）的元数据存储可以减轻这些可扩展性的挑战，而不损害高可用性的保证。然而，对于低规模部署（元数据可以容纳在单个机器上的内存中），这些基于DDBMS的系统通常比在单个机器上的内存中存储元数据的系统执行差一个数量级。这限制了这些分布式数据库方法的影响，因为它们目前仅适用于极端规模的文件系统。本文介绍了FileScale，一个三层体系结构，其中包括一个DDBMS的一部分，一个全面的方法来文件系统元数据管理。与以前的方法相比，FileScale在小规模上执行单机架构的可扩展性，同时随着文件系统元数据的增加实现线性可扩展性1。
File systems that store metadata on a single machine or via a shared-disk abstraction face scalability challenges, especially in contexts demanding the management of billions of files. Recent work has shown that employing shared-nothing, distributed database system (DDBMS) for metadata storage can alleviate these scalability challenges without compromising on high availability guarantees. However, for low-scale deployments -- where metadata can fit in memory on a single machine -- these DDBMS-based systems typically perform an order of magnitude worse than systems that store metadata in memory on a single machine. This has limited the impact of these distributed database approaches, since they are only currently applicable to file systems of extreme scale. This paper describes FileScale, a three-tier architecture that incorporates a DDBMS as part of a comprehensive approach to file system metadata management. In contrast to previous approaches, FileScale performs comparably to the single-machine architecture at a small scale, while enabling linear scalability as the file system metadata increases1.