Performance modelling and dynamic scheduling of geometric multigrid problems on heterogeneous supercomputing clusters

点击翻译按钮获取中文摘要

The supercomputer is an essential tool for solving computational problems far too large for a single computer, problems that tend to have great significance. A supercomputer is a large collection of individual computers networked together, and large problems are processed by dividing them into pieces and giving one to each computer to process in parallel. A well known example of such a large problem is weather forecasting as performed by the Met Office, who run a detailed model of the climate on their supercomputer in order to predict the weather over the following days. However supercomputers are very expensive to obtain, upgrade and operate - the cost of purchasing or upgrading a modern high-end supercomputer with tens of thousands of microprocessors is tens of millions of pounds and its electricity bill is millions of pounds. In order for a supercomputer owner to make informed financial decisions regarding their supercomputer it is crucial to understand the relationship between their particular computational needs and the configuration of their current supercomputer and of potential future configurations.This research is a partnership with the aerospace division of Rolls-Royce which designs and manufactures jet engines, and also with Intel who are a semiconductor manufacturer primarily known for their computer microprocessors, and the research has two goals. The first is to develop models of the relationship between the processing time of simulations in HYDRA, Rolls-Royce's engine for computing fluid dynamics on a supercomputer, and varying supercomputer configurations including that currently owned and operated by Rolls-Royce, and also to develop tools to assist with the development of those models. This entails modelling comprehensively how HYDRA divides simulations into pieces for parallel processing, then modelling how different hardware configurations fare in processing those pieces with the assistance of Intel. These models will provide two benefits: identifying inefficiencies in HYDRA, and allowing Rolls-Royce to evaluate potential upgrades to their supercomputer in regards to the effect on the performance of HYDRA.The second goal of this research is to develop a scheduling algorithm that uses the developed performance models of HYDRA simulations in order to allocate proportions of a supercomputer to simulation jobs as they are submitted in order to maximise the ratio between overall simulation work performed and electricity consumed. Such a scheduling algorithm will not be limited to HYDRA as it will be compatible with any simulation engine that has a performance model, thus it is of potential benefit to any business or research group that operates a supercomputer.