Performance Optimization using Multimodal Modeling and Heterogeneous GNN
Performance Optimization using Multimodal Modeling and Heterogeneous GNN
复制标题
DOI：
10.1145/3588195.3592984
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Akashnil Dutta;J. Alcaraz;Ali TehraniJamsaz;Eduardo César;A. Sikora;A. Jannesari
中科院分区：
文献类型：
--
作者：
Akashnil Dutta;J. Alcaraz;Ali TehraniJamsaz;Eduardo César;A. Sikora;A. Jannesari
高性能计算体系结构中日益增长的异构性和可配置性使得在这些系统上自动调整应用程序和运行时参数变得非常复杂。用户可以通过多种选项来配置参数。除了特定于应用程序的解决方案之外，一种常见的方法是使用通用搜索策略，这通常可能无法确定最佳配置，或者它们的收敛时间是一个重大障碍。因此，需要一种通用且高效的调优方法，这种方法可以轻松地扩展并适应各种调优任务。我们提出了一种调优并行代码区域的技术，这种技术足够通用，可以适应多个任务。在本文中，我们分析了基于ir的编程模型，以进行特定任务的性能优化。为此，我们提出了多模态图神经网络和自编码器（MGA）调谐器，这是一种基于多模态深度学习的方法，可适应异构图神经网络和去噪自编码器，用于建模作为单独模态的基于ir的代码表示。该方法被用作我们管道的一部分，用于建模语法、语义和结构感知的基于ir的代码表示，以调优并行代码区域/内核。我们对从PolyBench， Rodinia， STREAM, DataRaceBench, AMD SDK， NPB, NVIDIA SDK, Parboil， SHOC， LULESH, XSBench, RSBench, miniFE， miniAMR和Quicksilver基准测试和应用程序获得的OpenMP和OpenCL代码区域/内核进行了广泛的实验。我们将我们的多模态学习技术应用于(i)优化OpenMP循环中的线程数量，调度策略和块大小的任务，以及（ii）确定OpenCL内核异构设备映射的最佳设备。我们的实验表明，这种基于多模态学习的方法在几乎所有实验中都优于最先进的方法。
Growing heterogeneity and configurability in HPC architectures has made auto-tuning applications and runtime parameters on these systems very complex. Users are presented with a multitude of options to configure parameters. In addition to application specific solutions, a common approach is to use general purpose search strategies, which often might not identify the best configurations or their time to convergence is a significant barrier. There is, thus, a need for a general purpose and efficient tuning approach that can be easily scaled and adapted to various tuning tasks. We propose a technique for tuning parallel code regions that is general enough to be adapted to multiple tasks. In this paper, we analyze IR-based programming models to make task-specific performance optimizations. To this end, we propose the Multimodal Graph Neural Network and Autoencoder (MGA) tuner, a multimodal deep learning based approach that adapts Heterogeneous Graph Neural Networks and Denoising Autoencoders for modeling IR-based code representations that serve as separate modalities. This approach is used as part of our pipeline to model a syntax, semantics, and structure-aware IR-based code representation for tuning parallel code regions/kernels. We extensively experiment on OpenMP and OpenCL code regions/kernels obtained from PolyBench, Rodinia, STREAM, DataRaceBench, AMD SDK, NPB, NVIDIA SDK, Parboil, SHOC, LULESH, XSBench, RSBench, miniFE, miniAMR, and Quicksilver benchmarks and applications. We apply our multimodal learning techniques to the tasks of (i) optimizing the number of threads, scheduling policy and chunk size in OpenMP loops and, (ii) identifying the best device for heterogeneous device mapping of OpenCL kernels. Our experiments show that this multimodal learning based approach outperforms the state-of-the-art in almost all experiments.