CAREER: A Unified Framework for Multilevel Parallelization on Deep Computing Systems

Nikolopoulos，Dimitrios威廉和玛丽学院CCF-0346867这项研究活动涉及使用全局地址空间模型的显式多级并行化编程框架的设计，开发和部署。该框架的目标是即将到来的一代千万亿次级超级计算机，这些超级计算机基于具有多个级别的片上和片外并行性以及深存储器层次结构的架构基板。该研究解决了提高国家超级计算能力的生产率和利用率的需要，试图缩小计算机体系结构创新和并行编程实践之间的差距。这项活动的主要目标是减少所需的编程工作，映射算法并行分层硬件组件与异构手段并行执行，并协助程序员在派生分层并行应用程序的平衡设计。 在这项研究中调查的编程框架统一并行编程模型和方法，使并行代码更快地适应新的硬件平台。 同时，它形成了高性能编程的跨学科学生观众的教育和培训的基础。本研究的并行编程组件是围绕标准C++模板设计的，其中包含嵌套线程和迭代器的符号。并行性的符号与数据的模板化表示相结合，这允许在多个级别的并行执行结构上进行任意分区、共享和一致性控制。 虽然程序员强调嵌套并行，但多粒度线程和数据的编排和管理被委托给编译器和运行时系统。本研究探讨新的方法，通过垂直分析的程序控制粒度的多级并行。 周期性分析和选择性的运行时跟踪被用作在没有用户干预的情况下导出有效的数据分布和布局方案的手段。除了运行时分析，新的资源驱动的调度策略和新的微处理器功能，包括片上多线程，片上SIMD并行和投机执行，被纳入到并行化和程序优化过程。

Nikolopoulos, DimitriosCollege of William and MaryCCF-0346867This research activity involves the design, development and deploymentof a programming framework for explicit multilevel parallelization using a global address space model. The framework targets the upcoming generation of Petaflop-class supercomputers, which are based on architectural substrates with multiple levels of on-chip and off-chip parallelism and deep memory hierarchies. The research addresses the need for increased productivity and utilization of the National supercomputing power, by attempting to close the gap between computer architecture innovation and parallel programming practices. The main goal of this activity is to reduce the programming effort required for mapping algorithmic parallelism to hierarchical hardware components with heterogeneous means for parallel execution and assist programmers in deriving balanced designs of layered parallel applications. The programming framework investigated in this research unifies parallel programming models and methodologies and enables faster adaptation of parallel code to new hardware platforms. Concurrently, it forms a basis for education and training of interdisciplinary student audiences in high performance programming. The parallel programming component of this research is designed around standard C++ templates with notation for nested threads and iterators. The notations for parallelism are coupled with a templated representation of data, which allows for arbitrary partitioning, sharing, and coherence control at multiple levels of parallel execution constructs. While the programmer highlights nested parallelism, the orchestration and management of multigrain threads and data are delegated to the compiler and the runtime system. The research investigates novel methods for controlling the granularity of multilevel parallelism via vertical analysis of the program. Periodicity analysis and selective runtime tracing are used as means to derive effective data distribution and layout schemes without userintervention. Alongside runtime analysis, new resource-driven scheduling strategies and novel microprocessor features, including on-chip multithreading, on-chip SIMD parallelism and speculative execution, are incorporated into the parallelization and program optimization processes.