Performance modelling and dynamic scheduling of geometric multigrid problems on heterogeneous supercomputing clusters

超级计算机是解决单个计算机无法解决的计算问题的重要工具，这些问题往往具有重大意义。超级计算机是一大批联网的独立计算机的集合，处理大问题的方法是把它们分成几个部分，然后让每台计算机并行处理一个部分。这类大问题的一个众所周知的例子是英国气象局（Met Office）的天气预报，他们在超级计算机上运行了一个详细的气候模型，以便预测未来几天的天气。然而，超级计算机的获取、升级和操作都非常昂贵——购买或升级一台拥有数万个微处理器的现代高端超级计算机的成本高达数千万英镑，电费则高达数百万英镑。为了让超级计算机所有者对他们的超级计算机做出明智的财务决策，理解他们的特定计算需求与他们当前的超级计算机配置以及潜在的未来配置之间的关系是至关重要的。这项研究是与设计和制造喷气发动机的劳斯莱斯航空航天部门以及以计算机微处理器闻名的半导体制造商英特尔公司合作进行的，这项研究有两个目标。首先是建立在HYDRA （Rolls-Royce的超级计算机上计算流体动力学的引擎）中模拟处理时间与各种超级计算机配置（包括Rolls-Royce目前拥有和运营的超级计算机）之间关系的模型，并开发工具来协助这些模型的开发。这需要对HYDRA如何将模拟分成并行处理的部分进行全面建模，然后在英特尔的帮助下对不同硬件配置如何处理这些部分进行建模。这些模型将提供两个好处：识别HYDRA的低效率，并允许Rolls-Royce评估其超级计算机升级对HYDRA性能的影响。本研究的第二个目标是开发一种调度算法，该算法使用HYDRA模拟的开发性能模型，以便在超级计算机提交模拟作业时分配超级计算机的比例，以便最大限度地提高执行的总体模拟工作和消耗的电力之间的比例。这种调度算法将不局限于HYDRA，因为它将与任何具有性能模型的模拟引擎兼容，因此它对任何操作超级计算机的企业或研究小组都有潜在的好处。

The supercomputer is an essential tool for solving computational problems far too large for a single computer, problems that tend to have great significance. A supercomputer is a large collection of individual computers networked together, and large problems are processed by dividing them into pieces and giving one to each computer to process in parallel. A well known example of such a large problem is weather forecasting as performed by the Met Office, who run a detailed model of the climate on their supercomputer in order to predict the weather over the following days. However supercomputers are very expensive to obtain, upgrade and operate - the cost of purchasing or upgrading a modern high-end supercomputer with tens of thousands of microprocessors is tens of millions of pounds and its electricity bill is millions of pounds. In order for a supercomputer owner to make informed financial decisions regarding their supercomputer it is crucial to understand the relationship between their particular computational needs and the configuration of their current supercomputer and of potential future configurations.This research is a partnership with the aerospace division of Rolls-Royce which designs and manufactures jet engines, and also with Intel who are a semiconductor manufacturer primarily known for their computer microprocessors, and the research has two goals. The first is to develop models of the relationship between the processing time of simulations in HYDRA, Rolls-Royce's engine for computing fluid dynamics on a supercomputer, and varying supercomputer configurations including that currently owned and operated by Rolls-Royce, and also to develop tools to assist with the development of those models. This entails modelling comprehensively how HYDRA divides simulations into pieces for parallel processing, then modelling how different hardware configurations fare in processing those pieces with the assistance of Intel. These models will provide two benefits: identifying inefficiencies in HYDRA, and allowing Rolls-Royce to evaluate potential upgrades to their supercomputer in regards to the effect on the performance of HYDRA.The second goal of this research is to develop a scheduling algorithm that uses the developed performance models of HYDRA simulations in order to allocate proportions of a supercomputer to simulation jobs as they are submitted in order to maximise the ratio between overall simulation work performed and electricity consumed. Such a scheduling algorithm will not be limited to HYDRA as it will be compatible with any simulation engine that has a performance model, thus it is of potential benefit to any business or research group that operates a supercomputer.