Collaborative Research: Actively Managing Data Movement with Models - Taming High Performance Data Communications in Exascale Machines

大型科学应用具有复杂的通信需求，包括机器内点对点和全球通信、跨机器检查点以及用于在线验证和远程数据显示的数据输出。多学科研究的科学模型组合进一步增加了这种复杂性。未来亿级代码中数据通信的这种多用途、丰富和动态的特性是本研究解决的第一个挑战。此外，考虑到未来计算机芯片的多核特性以及可能存在的专用硬件加速器核心，应用层通信和I/O面临着一组日益复杂的片上、跨节点和跨机器互连。未来亿级机器中存在的物理通信基础设施的复杂性质是本研究解决的第二个挑战。总而言之，未来亿级科学发现应用程序的开发者面临的问题是如何有效地管理他们通信需求的复杂性，同时保护他们最关键的“核心”通信不受干扰。该项目将为未来艾级代码中执行的数据通信开发更高级别的显式模型。这些称为C-Models的模型将描述和实现为I/O执行的通信，用于在线分析、存储和可视化，以及用于跨耦合应用程序代码的数据移动，还将捕获所有上述内容所隐含的数据移动与每个单独应用程序固有的内部数据通信之间的交互。这种通信复杂性的抽象和封装是驯服未来亿级应用程序复杂性的关键。C-Model基础设施还将帮助保护这些应用程序的关键核心通信组件不受干扰，帮助最大限度地提高这些应用程序在领先级别机器上的性能。

Large scientific applications have complex communication needs, including intra-machine point-to-point and global communications, cross-machine checkpointing, and data outputs for online validation and remote data display. Coupled scientific models for multidisciplinary investigations further add to this complexity. This multi-purpose, rich, and dynamic nature of data communications in future exascale codes presents the first challenge addressed by this research.Further, given the many-core nature of future computer chips and the likely presence of specialized hardware accelerator cores, application-level communications and I/O face an increasingly complex set of on-chip, cross-node, and cross-machine interconnects. The complex nature of the physical communication infrastructures present in future exascale machines is the second challenge addressed by this research. In summary, the problem facing developers of future exascale applications for scientific discovery is how to effectively manage the complexity of their communication needs while protecting their most critical `core' communications from perturbation.This project will develop higher level, explicit models for the data communications performed in future exascale codes. These models, called C-Models, will describe and implement the communications performed for I/O for purposes of online analysis, storage, and visualization, and for data movements across coupled application codes, and will also capture the interaction of the data movements implied by all of the above with the internal data communications inherent to each single application. This abstraction and encapsulation of communication complexity is key to taming the complexity of future exascale applications. The C-Model infrastructure will also help protect the critical core communication component of these applications from perturbation, helping to maximize the performance of these applications on leadership-class machines.