CAREER: A Unified Framework for Multilevel Parallelization on Deep Computing Systems

这项研究活动涉及使用全局地址空间模型进行显式多层并行化的编程框架的设计、开发和部署。该框架的目标是即将到来的千万亿次级超级计算机，这些超级计算机基于具有多层片内和片外并行性和深度内存层次结构的架构基板。该研究通过试图缩小计算机架构创新与并行编程实践之间的差距，解决了提高生产力和利用国家超级计算能力的需求。该活动的主要目标是减少将算法并行性映射到具有并行执行异构手段的分层硬件组件所需的编程工作，并帮助程序员派生分层并行应用程序的平衡设计。本研究中研究的编程框架统一了并行编程模型和方法，使并行代码能够更快地适应新的硬件平台。同时，它形成了教育和训练跨学科的学生观众在高性能节目的基础。本研究的并行编程组件是围绕带有嵌套线程和迭代器符号的标准c++模板设计的。并行性的符号与数据的模板表示相结合，这允许在并行执行构造的多个级别上进行任意分区、共享和一致性控制。当程序员强调嵌套并行性时，多粒度线程和数据的编排和管理被委托给编译器和运行时系统。通过对程序的纵向分析，研究了控制多层并行度粒度的新方法。采用周期性分析和选择性运行时跟踪方法，在不需要用户干预的情况下推导出有效的数据分布和布局方案。除了运行时分析，新的资源驱动调度策略和新的微处理器特性，包括片上多线程，片上SIMD并行和推测执行，被纳入并行化和程序优化过程。

Nikolopoulos, DimitriosCollege of William and MaryCCF-0346867This research activity involves the design, development and deploymentof a programming framework for explicit multilevel parallelization using a global address space model. The framework targets the upcoming generation of Petaflop-class supercomputers, which are based on architectural substrates with multiple levels of on-chip and off-chip parallelism and deep memory hierarchies. The research addresses the need for increased productivity and utilization of the National supercomputing power, by attempting to close the gap between computer architecture innovation and parallel programming practices. The main goal of this activity is to reduce the programming effort required for mapping algorithmic parallelism to hierarchical hardware components with heterogeneous means for parallel execution and assist programmers in deriving balanced designs of layered parallel applications. The programming framework investigated in this research unifies parallel programming models and methodologies and enables faster adaptation of parallel code to new hardware platforms. Concurrently, it forms a basis for education and training of interdisciplinary student audiences in high performance programming. The parallel programming component of this research is designed around standard C++ templates with notation for nested threads and iterators. The notations for parallelism are coupled with a templated representation of data, which allows for arbitrary partitioning, sharing, and coherence control at multiple levels of parallel execution constructs. While the programmer highlights nested parallelism, the orchestration and management of multigrain threads and data are delegated to the compiler and the runtime system. The research investigates novel methods for controlling the granularity of multilevel parallelism via vertical analysis of the program. Periodicity analysis and selective runtime tracing are used as means to derive effective data distribution and layout schemes without userintervention. Alongside runtime analysis, new resource-driven scheduling strategies and novel microprocessor features, including on-chip multithreading, on-chip SIMD parallelism and speculative execution, are incorporated into the parallelization and program optimization processes.