CAREER: Avoiding Achilles' Heel in Exascale Computing with Distributed File Systems

Exascale（即每秒1018次运算）计算机将能够揭开重大的科学奥秘，涵盖许多领域（例如天气建模，国家安全，能源和药物发现）。预测是，到2019年将达到亿亿级，拥有数百万个计算节点和数十亿个执行线程。高端计算（HEC）中当前最先进的存储，其中存储与计算节点隔离并通过网络（例如并行文件系统）连接，将不会随着并发性的预期指数增长而扩展。在exascale中，高并发级别的基本功能（例如引导，检查点，元数据/数据访问）将受到性能差的影响，并且与系统平均故障时间（以小时为单位）相结合，将导致性能崩溃。研究人员设想未来的HEC系统将在每个计算节点上设计有非易失性存储器，并且每个节点都积极参与元数据和数据管理。这项工作旨在：1）设计、分析和实现针对HEC优化的分布式数据结构（D3），以用于分布式元数据管理; 2）设计、分析和实现针对重要的高性能计算（HPC）以及多任务计算（MTC）工作负载的子集优化的分布式文件系统（FDFS），并且可扩展到数百万个节点;以及3）评估真实的工作负载、应用程序和高达数十亿次的模拟。这项工作的结果有可能使百亿亿次计算更容易处理，几乎涉及HEC的所有学科，推动国家层面的科学发现和经济发展。HEC知识库将扩展到商品系统，因为最快的机器通常在五到七年内成为主流系统。这项工作也可以打开大门，研究在激进的并行编程范例（如MTC），依赖于可扩展的存储基础设施。

Exascale (i.e. 1018 operations/sec) computers will enable the unraveling of significant scientific mysteries, covering many domains (e.g. weather modeling, national security, energy, and drug discovery). Predictions are that exascales will be reached in 2019, with millions of compute-nodes and billions of threads of execution. The current state-of-the-art storage in high-end computing (HEC), in which storage is segregated from compute-nodes and connected by a network (e.g. parallel filesystems), will not scale with the expected exponential growth in concurrency. At exascales, basic functionality (e.g. booting, check-pointing, metadata/data access) at high concurrency levels will suffer poor performance, and combined with system mean-time-to-failure in hours, will lead to a performance collapse. The investigator envisions future HEC systems to be designed with non-volatile memory on every compute node, and every node to actively participate in the metadata and data management. This work aims to: 1) design, analyze, and implement a distributed data structure (D3) optimized for HEC, to be used for distributed metadata management; 2) design, analyze, and implement a distributed filesystem (FDFS) optimized for a subset of important high-performance computing (HPC) as well as many-task computing (MTC) workloads, and scalable to millions of nodes; and 3) evaluate work with real workloads, applications, and simulations up to exascales. The results of this work has the potential to make exascale computing more tractable, touching virtually all disciplines in HEC, fueling scientific discovery and economic development at the national level. The HEC knowledgebase will extend into commodity systems as the fastest machines generally become mainstream systems in five to seven years. This work can also open doors for research in radical parallel programming paradigms (e.g. MTC) that rely on scalable storage infrastructure.