Hybrid MPI-OpenMP Parallelism in the ONETEP Linear-Scaling Electronic Structure Code: Application to the Delamination of Cellulose Nanofibrils.
Hybrid MPI-OpenMP Parallelism in the ONETEP Linear-Scaling Electronic Structure Code: Application to the Delamination of Cellulose Nanofibrils.
复制标题
DOI：
10.1021/ct500686r
复制
发表时间：
2014-10
影响因子：
5.5
通讯作者：
Karl A. Wilkinson;N. Hine;Chris-Kriton Skylaris
中科院分区：
文献类型：
--
作者：
Karl A. Wilkinson;N. Hine;Chris-Kriton Skylaris
我们提出了一个混合的MPI-OpenMP实现的线性尺度密度泛函理论在ONETEP代码。我们说明了它的性能范围内的高性能计算（HPC）平台，包括共享内存节点与快速互连。我们的工作重点是将OpenMP并行性应用于主导计算负载的例程，并尝试在可能的情况下并行化MPI中已并行化的循环中的不同循环。这包括3D FFT框运算、稀疏矩阵代数运算、积分计算和埃瓦尔德求和。虽然底层的数值方法没有改变，但这些发展代表了ONETEP中用于跨CPU内核分配工作负载的算法的重大变化。新的混合代码表现出大大改善了强大的缩放相对于MPI的唯一代码，并允许计算的核心原子的比例要高得多。这些发展的结果是在一个显着更短的时间解决方案比可能单独使用MPI和促进应用程序的ONETEP代码的系统比以前可行的。我们用含有41,907个原子的淀粉样蛋白原纤维三聚体的基准计算来说明这一点。我们使用的代码来研究纤维素纳米原纤维的分层机制时，进行超声处理，这是一个过程，这是由大量的相互作用，共同确定的原纤维的结构特性控制。这些模拟需要进行许多能量评估，由于这些系统包含多达21，276个原子，如果没有这里描述的发展，这是不可行的。
We present a hybrid MPI-OpenMP implementation of Linear-Scaling Density Functional Theory within the ONETEP code. We illustrate its performance on a range of high performance computing (HPC) platforms comprising shared-memory nodes with fast interconnect. Our work has focused on applying OpenMP parallelism to the routines which dominate the computational load, attempting where possible to parallelize different loops from those already parallelized within MPI. This includes 3D FFT box operations, sparse matrix algebra operations, calculation of integrals, and Ewald summation. While the underlying numerical methods are unchanged, these developments represent significant changes to the algorithms used within ONETEP to distribute the workload across CPU cores. The new hybrid code exhibits much-improved strong scaling relative to the MPI-only code and permits calculations with a much higher ratio of cores to atoms. These developments result in a significantly shorter time to solution than was possible using MPI alone and facilitate the application of the ONETEP code to systems larger than previously feasible. We illustrate this with benchmark calculations from an amyloid fibril trimer containing 41,907 atoms. We use the code to study the mechanism of delamination of cellulose nanofibrils when undergoing sonification, a process which is controlled by a large number of interactions that collectively determine the structural properties of the fibrils. Many energy evaluations were needed for these simulations, and as these systems comprise up to 21,276 atoms this would not have been feasible without the developments described here.