CAREER: Harnessing Hybrid Computing Resources in PetaScale Computing and Beyond

未来的亿级计算系统预计将超过10万个节点，并包含通过将多核处理器与加速器处理器相结合而创建的丰富的异类计算资源组合。然而，当前的应用程序还不能利用超过几万个这样的节点，也不能有效地管理这种异类资源组合来平衡工作负载。此外，混合或多范例系统中的这种不同种类的资源混合为程序员提供了利用算法中各种类型的并行性的新的、但具有挑战性的机会。这项职业研究将为包括多核处理器和加速器的混合计算系统的性能建模和分析制定一个包容的分层框架。该框架包括用于快速评估应用程序和混合系统之间的适应性匹配的粗粒度模型和用于预测执行时间和工作负载平衡分析的细粒度模型。这些模型将有助于研究人员和科学家开发、优化和维护科学代码。研究项目包括通过对多核和加速器处理器上的应用程序优化和执行的严格研究，形成应用程序和体系结构特征的分类。研究的重点是跨混合异质节点的应用程序的节点间并行性和多范例节点上的节点内并行性。这些混合系统研究阐明了层次框架的粗粒度模型和细粒度模型中包含的应用程序和体系结构特征。对于本研究中未包括的其他应用领域，对优化方法(资源比例优化、负载均衡和内存管理)和模型框架进行了定性评估。该项目还将这些知识转化为适合本科生和研究生课程课堂使用的教育模块。

Future ExaScale computing systems are expected to exceed a hundred-thousand nodes and contain a rich mix of heterogeneous computing resources created by coupling multi-core processors with accelerator processors. However, current applications are not yet able to exploit more than a few tens of thousands of these nodes, or effectively manage this heterogeneous mix of resources to balance workloads. Moreover, this heterogeneous mixture of resources in hybrid or multi-paradigm systems presents programmers with new, but challenging opportunities for exploiting the various types of parallelism within algorithms. This CAREER research will formulate an inclusive hierarchical framework for performance modeling and analysis of hybrid computing systems that include multi-core processors and accelerators. The framework includes a coarse-grained model for rapid assessment of the fitness match between the application and the hybrid system and a fine-grained model for predicting execution time and workload balance analysis. These models will be useful for researchers and scientists developing, optimizing, and maintaining scientific codes. The research project includes forming a taxonomy of application and architecture characteristics through a rigorous study of application optimization and execution on multi-core and accelerator processors. The study focuses on the inter-node parallelism of the applications across a mix of heterogeneous nodes and the intra-node parallelism on the multi-paradigm nodes. These hybrid system studies elucidate the application and architecture characteristics included in the coarse and fine-grained models of the hierarchical framework. The optimization methods (resource ratio optimization, load balancing and memory management) and the model framework are qualitatively assessed for other application domains not included in this study. The project also transitions this knowledge into educational modules suitable for classroom use in the undergraduate and graduate curriculum.