CSR: Small: Adaptively Applying Data-Driven Execution Mode to Remove I/O Bottleneck for Data-Intensive Computing

越来越常见的多核/众核CPU架构对于加速程序是有效的。只有在保持足够的并行性时才执行。对于数据密集型程序，增加的并行性会严重损害I/O效率：当一个顺序程序被并行化时，不仅计算，而且与它们相关的I/O操作也可以分布在多个进程中。由于进程的执行顺序通常由调度程序在运行时确定，因此进程的相对进度是不确定的，进程发出I/O请求的顺序也是不确定的。正是这种I/O的不确定性，可以大大损害I/O的效率，从而程序的性能，通过否定的优点，并行执行。为了解决这个问题，PI提出了一个设施，无论是在操作系统内核或运行时，streamlinethe服务的I/O请求从不同进程的并行程序。与传统的提高I/O性能的技术不同的是，通过I/O感知的进程调度来协调I/O请求的发出，以提高I/O密集型多线程和MPI程序的I/O请求的本地性，如果成功的话，该研究将为以数据为中心的计算引入一种破坏性技术，以有效地缓解I/O瓶颈。该项目还为学生提供了丰富的研究培训机会，特别是密歇根州东南部地区代表性不足的少数民族学生，以帮助缓解具有并行计算和存储系统专业知识的IT专业人员的短缺。

The increasingly common multi-core/many-core CPU architecturesare effective for accelerating programsâ?? execution only whensufficient parallelism is maintained. For data-intensiveprograms the increased parallelism can severely compromise I/Oefficiency: when a sequential program is parallelized, not onlycomputations, but also the I/O operations associated with them, canbe distributed among multiple processes. Because the executionorder of the processes is usually determined by the scheduler atruntime, the relative progress of processes is nondeterministicand the order in which the processes issue their I/O requests isaccordingly nondeterministic. It is this I/O nondeterminism thatcan substantially compromise I/O efficiency, and thus programperformance, by negating the advantages of parallel execution.To address this problem the PI proposes a facility built eitherin the operating system kernel or in the runtime to streamlinethe service of I/O requests from different processes of aparallel program. The major distinction from conventionaltechniques for improving I/O performance is in the coordinationof I/O request issuance, via I/O-aware process scheduling, toimprove the locality of these requests for I/O-intensivemultithreaded and MPI programs.If successful, the proposed research would introduce a disruptivetechnique for data-centric computing to effectively relieve theI/O bottleneck. This project also provides abundant researchtraining opportunities for students, especially under-representedminority students in the southeast Michigan area, to help relievethe shortage of IT professionals with expertise in parallelcomputing and storage systems.