Parallel Scalability of Elliptic Solvers in Weather and Climate Prediction

英国气象局是世界上天气和气候预测的领导者之一，世界各地的许多其他中心都使用气象局的全球预报模型来驱动他们各自的局部区域模型。然而，由于全球模型有限的空间分辨率和局部区域模型引入的额外误差，许多短尺度现象以及重要的长期动力学仍然难以准确预测。具有超过10^5个核心的新型计算架构提供了一个突破这些界限的机会，并使英国气象局保持在发展的最前沿。数十年的数值天气和气候预测经验使人们对大气流动固有的核心动力学及其稳定和准确的数值近似有了很好的理解。正如电话会议中所概述的，气象局的统一模型使用经纬度网格，并在多达1000个核心的并行计算机上实现了高效率。然而，在极点的（人工）网格聚类使得这些网格对于大规模计算是不切实际的，因此NERC计划的核心任务之一是寻找合适的替代网格。有几项单独的提案涉及这一问题。然而，控制大气流动的方程形成了一个依赖于时间的微分方程系统，该系统在地球仪上的任何地方都强烈耦合解（著名的“蝴蝶效应”）。目前大多数大气动力学模式使用半隐式时间离散方案，在每个时间步提供一些全球耦合的方程。这可以防止系统变得不稳定，因此它允许比完全显式格式更大的时间步长，其中不包括全局耦合。由于预测的成本与时间步长的数量成正比，因此允许更大时间步长（具有令人满意的准确性）的方案似乎更可取。但这些好处是有代价的，特别是在大规模问题和大规模并行架构的背景下。一个椭圆系统的压力必须解决在每个时间步，导致一个非常大的，病态的代数系统，其解决方案是难以有效地并行化。有两个主要因素，使缩放这个椭圆解决大的问题大小和大处理器数量困难：算法的可扩展性和并行可扩展性。由于椭圆方程的解算子耦合了全局压力，因此只有在不同分辨率的网格上使用离散化层次的多级迭代求解器才允许成本的最佳线性增长（算法可扩展性）。但在大规模并行计算环境中，全局通信成本很高，因此有必要很好地实现这些求解器，将大部分通信保持在本地，以确保计算成本继续优化扩展到100 K或更多处理器（并行可扩展性）。该提案解决了这个问题，因此将有助于对气象局未来动态核心的设计做出最佳决策，从而保证英国在这一关键的社会/技术挑战中的竞争力。到目前为止，半隐式格式在大气流动中还没有实现最佳的可扩展性，但是项目合作伙伴IWR Heidelberg和Lawrence Livermore National Lab在更简单的模型椭圆问题上的成功表明这是可能的。PI多年来在各种应用领域的最新架构上获得椭圆解算器的最佳可扩展性方面的经验，最值得注意的是在经纬度网格上离散大气流的椭圆问题多达256个核心，以及他作为世界领先的多层迭代椭圆解算器理论分析师之一的地位，以及他与该领域其他世界领先团体的联系，这意味着他是理想的装备来实现这一目标。

The UK Met Office is one of the world leaders in weather and climate prediction, and the Met Office's global forecast model is used by many other centres worldwide to drive their individual local area models. However, many short scale phenomena as well as important longterm dynamics are still difficult to predict accurately due to the limited spatial resolution of global models and the additional errors introduced by local area models. Novel computing architectures with more than 10^5 cores provide a chance to push these boundaries and to keep the UK Met Office at the forefront of developments. Decades of experience with numerical weather and climate prediction have produced a good understanding of the core dynamics inherent in atmospheric flow and of their stable and accurate numerical approximations. As outlined in the call, the Met Office's Unified Model uses lattitude-longitude grids and achieves high efficiency on parallel computers with up to 1000 cores. However, (artificial) grid clustering at the poles renders these grids impractical for large-scale computations, and so one of the core tasks in this NERC Programme is the search for suitable alternative grids. Several separate proposals address this issue. However, the equations governing atmospheric flow form a time-dependent system of differential equations which strongly couple the solution everywhere on the globe (the famous "butterfly effect"). Most current atmospheric dynamics models use semi-implicit time discretisation schemes which provide some global coupling of the equations at each time step. This prevents the system from becoming unstable and as a consequence it allows for larger time steps than fully explicit schemes, which include no global coupling. Since the cost of the forecast is proportional to the number of time steps, a scheme that allows for larger time steps (with satisfactory accuracy) seems preferable. But these benefits come at a price, especially in the context of large-scale problems and on massively parallel architectures. An elliptic system for the pressure has to be solved in each time step, leading to a very large, ill-conditioned algebraic system, the solution of which is difficult to parallelise efficiently. There are two main factors that make the scaling of this elliptic solve to large problem sizes and to large processor numbers difficult: algorithmic scalability and parallel scalability. Since the solution operator for the elliptic equation couples the pressures globally, only multilevel iterative solvers which use a hierarchy of discretisations on grids of varying resolution allow optimal, linear growth in cost (algorithmic scalability). But in a massively parallel computing environment, where global communication is costly, it is necessary to implement these solvers well, keeping most of the communication local, to ensure that the computational cost continues to scale optimally to 100K or more processors (parallel scalability).This proposal addresses this problem and will thus facilitate the best possible decisions on the design of the Met Office's future dynamical core, thus guaranteeing the UK's competitiveness in this key societal/technological challenge. An optimal scalability of semi-implicit schemes has not been achieved in atmospheric flow up to now, but success of the Project Partners, IWR Heidelberg and Lawrence Livermore National Lab, on simpler model elliptic problems shows that it is possible. The PIs experience over the years in obtaining optimal scalability of elliptic solvers on the most current architectures in various application areas, most notably for elliptic problems from atmospheric flow discretised on latitude-longitude grids up to 256 cores, as well as his status as one of the world's leading theoretical analysts of multilevel iterative elliptic solvers and his links to other world leading groups in this field, mean that that he is ideally equipped to achieve this goal.