Data Distribution and Routing in Shared and Distributed Memory Parallel Computers

点击翻译按钮获取中文摘要

Distribution of data of a parallel computation such that data are accessed free of memory bank and interconnection network conflicts, is a critical issue for efficient use of shared and distributed memory computers. Conflicts in memory banks and interconnection networks increase inter-processor communication delays, which in turn adversely affect the execution times of parallel computations and cost-performance ratio of parallel computers. This research investigates hardware and software solutions, which can be applied in the design of parallel architectures for consistent high performance for a variety of parallel computations and efficient execution of tasks in a transparent manner, even in the presence of faults. In the first part of the research, recent work on hardware techniques, called skewed storage schemes applicable at the time of task execution, will be expanded upon. The new techniques will be useful to design and use efficiently fault-tolerant and gracefully degrading memories, multidimensional memories, and memory hierarchies for large data accesses in parallel computations. Suitability of the techniques will be investigated with theoretical analysis, experiments on current parallel computers, and simulations. In the second part of the research, software techniques, applicable at compile-time, will be investigated to utilize and assist hardware techniques for partitioning and allocation of data, including heavily accessed and hotspot type data. Effectiveness of these results will be evaluated experimentally.