CSR: Small: Automatic Storage and Network Contention Management for Large-scale High-performance Computing Systems

CSR：小型：大规模高性能计算系统的自动存储和网络争用管理

• 批准号: 1528179
• 负责人: Darrell Long
• 金额: $ 45万
• 依托单位: University of California-Santa Cruz
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1528179&HistoricalAwards=false
• 关键词: CSR; Small; Automatic; Storage; Network

High performance computing is essential to science, industry, and the environment, from resource exploration to the design of the next generation of consumer electronics. These high performance computer systems are among the most complex and expensive computer systems and require that their resources be used in the most efficient manner. Many of the applications that utilize high performance computing are data-intensive, and storage system performance is a crucial aspect of system performance. However, storage systems are notoriously sensitive to contention caused by competition among storage clients for limited bandwidth and disk access. This is a significant problem for shared storage systems. This project provides an automatic storage contention alleviation and reduction system (ASCAR) for large-scale high-performance storage to increase bandwidth utilization and fairness of resource allocation. ASCAR uses machine learning methods combined with several heuristics to discover the fittest control strategy. It is a highly scalable and fully automatic storage contention and congestion management system, which can improve the efficiency of both legacy and new systems, with no need to change either server hardware/software or existing applications. ASCAR regulates I/O traffic from the client side using a rule based algorithm. It employs a shared-nothing design and requires no runtime coordination between clients or with a central coordinator whatsoever, because runtime coordination is slow and unscalable. The effectiveness of ASCAR relies on the quality of traffic control. The research team has designed a prototype algorithm, the SHAred-nothing Rule Producer (SHARP), which produces rules in an unsupervised manner by systematically exploring the solution space of possible designs. Starting from one initial rule, SHARP uses heuristics similar to random-restart hill climbing to find the optimal parameters without the need for an exhaustive search. ASCAR monitors the workloads running on the system and uses several heuristics to pick up the fittest rules. It is clear that computer systems are getting ever more sophisticated, and human-lead empirical-based approach towards system optimization is not the most efficient way to realize the full potential of these modern, complex, high performance computing systems. This research brings machine learning, artificial intelligence, and big data methods to systems research and could lead to a very low cost I/O performance increase for a wide range of systems.

从资源勘探到下一代消费电子产品的设计，高性能计算对科学、工业和环境至关重要。这些高性能计算机系统是最复杂和昂贵的计算机系统之一，并且要求以最有效的方式使用它们的资源。许多利用高性能计算的应用程序都是数据密集型的，存储系统性能是系统性能的一个关键方面。然而，众所周知，存储系统对由存储客户端之间对有限带宽和磁盘访问的竞争引起的争用非常敏感。这对于共享存储系统来说是一个严重的问题。本计画针对大规模高效能储存环境，提供一个自动储存争用缓解与减少系统（ASCAR），以提升频宽利用率与资源分配的公平性。ASCAR使用机器学习方法结合几种算法来发现最合适的控制策略。它是一个高度可扩展和全自动的存储争用和拥塞管理系统，可以提高旧系统和新系统的效率，而无需更改服务器硬件/软件或现有应用程序。ASCAR使用基于规则的算法调节来自客户端的I/O流量。它采用了无共享的设计，不需要客户端之间的运行时协调，也不需要与中央协调器进行任何协调，因为运行时协调速度很慢，而且不可扩展。ASCAR的有效性依赖于交通控制的质量。研究团队设计了一个原型算法，SHARP规则生成器（SHARP），它通过系统地探索可能设计的解决方案空间，以无监督的方式生成规则。从一个初始规则开始，SHARP使用类似于随机重启爬山的算法来找到最佳参数，而不需要穷举搜索。ASCAR监控系统上运行的工作负载，并使用多种启发式方法来选择最合适的规则。很明显，计算机系统变得越来越复杂，而人类主导的基于化学的系统优化方法并不是实现这些现代，复杂，高性能计算系统的全部潜力的最有效方法。这项研究将机器学习、人工智能和大数据方法引入系统研究，并可能为各种系统带来非常低成本的I/O性能提升。