AtmosFOAM parallel scaling on HECToR and New Test Cases which expose Grid Scale Oscillations

过去几十年中一些最好的天气和气候预测模型在未来几十年的计算机架构上似乎不能很好地工作。计算机功率的进一步增加需要伴随着减少的电力消耗，并且这沿着硬件成本，意味着未来几代的高性能计算机将包括数万或数十万更便宜、功能更弱的处理核心，诸如计算机游戏所使用的。预测模型需要以不同的方式设计，以便计算可以有效地分布在许多功能较弱的核心上。在设计在下一代计算机上表现良好的新预测模型时，必须独立比较模型的不同方面，并且必须能够在组装整个模型之前严格测试简化的模型版本。这两个方面将在本项目中分别涉及：（1）建议进一步开发一个简化的全球大气模型：AtmosFOAM，运行在英国高性能计算机HECToR的10，000个处理核心上。AtmosFOAM是使用已经在HECToR上运行的OpenFOAM软件库编写的。使用OpenFOAM编写的其他模型已被证明可以在HECToR上扩展到1，000个或更多核心。它的目标是达到15公里的水平空间分辨率，50个垂直水平意味着大约1.5亿个计算点，每个核心上有10到1.5万个点。这个问题大小应该足够大，可以扩展到10，000个核心。并行缩放的两个瓶颈（由英国气象局确定）是半隐式时间步进，它允许更长的时间步长，但需要所有处理核心之间的同步，全球通信，以及在北极和南极附近恒定经度线收敛的解决方案的平滑。半隐式时间步长很重要，因为它意味着时间步长不必短到足以分辨声波。然而，如果在水平方向上使用显式时间步进，则需要较少的处理器通信，但需要较小的时间步长。在某种程度上，回到20世纪80年代的建模技术并使用时间步进将变得有益，这在水平方向上是明确的。这个项目将有助于找出这种变化的数值算法时，必须通过比较的精度和并行性能的两个时间步进计划在一个模型，否则是相同的。解决恒定经度的线向两极收敛的问题的一个方法是使用球面的六边形-二十面体网格而不是经纬度网格。这种镶嵌更像一个足球，但它将球体分成更多的六边形。将使用相同的模型在两种网格类型上比较并行性能和精度，以便对两种网格进行清晰的比较。所有代码开发都将在Gnu公共许可证（GPL）下免费提供和开源。(2)研究的一个独立部分将是为简化的大气模型创建新的测试案例，这些模型模拟了真实的世界中发生的非常小尺度的强烈活动，而这种活动无法在全球模型中解决。例如，雷暴和非常陡峭的山脉不能完全解决，但可以显着影响较大的规模。在这些新的测试用例中，将在方程中添加仅在单个点模拟这些影响的项。这些强迫将在不适当的模型中产生网格尺度振荡或其他不切实际的行为，否则这些行为可能在其他现有的简单测试用例中表现良好。这可以防止业务中心采用不适当的模式。

Some of the best weather and climate forecasting models of the past few decades appear not to work well on the computer architectures of the coming decades. Further increases in computer power need to be accompanied by reduced electricity consumption and this, along with hardware cost, means that future generations of high performance computers will consist of a tens or hundreds of thousands of cheaper, less powerful processing cores, such as are used by computer games. Forecasting models need to be designed differently so that the computation can be distributed efficiently over so many less powerful cores. In designing new forecasting models that will perform well on next generation computers it is essential to compare different aspects of the model independently and it is essential to be able to test simplified model versions rigorously before the entire model is assembled. These two aspects will be covered separately in this project: (1) It is proposed to further develop a simplified model of the global atmosphere: AtmosFOAM, to run on 10,000 processing cores of the UK high performance computer, HECToR. AtmosFOAM is written using the OpenFOAM software library which has already been run on HECToR. Other models written using OpenFOAM have been shown to scale well to 1,000 or more cores on HECToR. It is aimed to reach a horizontal spatial resolution of 15km with 50 vertical levels meaning around 150 million computational points with 10 to 15 thousand points on each core. This problem size should be large enough to scale to 10,000 cores. Two of the bottlenecks to parallel scaling (identified by the Met Office) are semi-implicit time stepping which allows longer time steps but requires synchronised, global communication across all processing cores, and the smoothing of the solutions near the north and south poles where the lines of constant longitude converge. Semi-implicit time stepping is important because it means that the time step does not have to be short enough to resolve sound waves. However if explicit time stepping is used instead in the horizontal, less processor communication is needed but smaller time steps are needed. At some point it will become beneficial to go back to the modelling techniques of the 1980s and use time stepping which is explicit in the horizontal. This project will help to find out when this change in numerical algorithm must be made by comparing the accuracy and parallel performance of the two time stepping schemes in a model which is otherwise identical. A solution to the problem of the convergence of the lines of constant longitude towards the poles is to use a hexagonal-icosahedral grid of the sphere instead of a latitude-longitude grid. This tessellation is more like a football but it divides the sphere into many more hexagons. Parallel performance and accuracy will be compared on the two grid types using a model which is otherwise identical in order to give a clean comparison of the two grids. All code developments will be freely available and open source under the Gnu public licence (GPL). (2) An independent part of the research will be to create new test cases for simplified models of the atmosphere which mimic the strong activity at very small scales that occur in the real world which cannot be resolved in a global model. For example thunderstorms and very steep mountains cannot be fully resolved but can dramatically influence the larger scales. In these new test cases, terms will be added to the equations which mimic these influences at just a single point. These forcings will generate grid scale oscillations or other unrealistic behaviour in inadequate models which otherwise might appear to work well for other existing simple test cases. This could prevent inadequate models from being adopted by operational centres.