Collaborative Research: Ultra-Scalable System Software and tools for Data-Intensive Computing

该项目需要研究和开发，并解决超大规模并行机的软件和工具问题，特别是针对可扩展的I/O、存储和内存层次结构。基本前提是，为了实现极高的可扩展性，对用于扩展数据访问和I/O的传统(顺序)接口和技术的增量更改或适应将不会成功，因为它们基于对并行性、同步和数据共享模式的悲观和保守假设。我们将开发创新的技术来优化数据访问，这些技术利用对高级访问模式(“意图”)的理解，并通过运行时层使用该信息来支持在不同级别上优化和减少/消除锁定和同步。拟议的机制将允许不同的软件层相互交互/协作。具体地说，软件堆栈中的上层提取高级访问模式信息，并将其传递给堆栈中的较低层，后者进而利用它们来实现超可伸缩性。具体地说，该项目的主要目标是：(1)用于在运行时提取数据访问模式和数据流的技术、工具和软件；(2)用于跨不同层传递访问模式以进行优化的接口和策略；(3)在适当的层中实现这些技术以减少或消除同步和锁定；(4)利用访问模式来降低底层存储系统的功耗和冷却需求的运行时技术和工具；以及(5)开发使用主动存储进行数据分析和过滤的接口和软件

This project entails research and development and to address the software and tools problems fro ultra-scale parallel machines, especially targeted for scalable I/O, storage and memory hierarchy. The fundamental premise is that to achieve extreme scalability, incremental changes or adaptation of traditional (extension of sequential) interfaces and techniques for scaling data accesses and I/O will not succeed, because they are based on pessimistic and conservative assumptions of parallelism, synchronization, and data sharing patterns. We will develop innovative techniques to optimize data access that utilize the understanding of high-level access patterns ("intent"), and use that information through runtime layers to enable optimizations and reduction / elimination of locking and synchronization at different levels. The proposed mechanisms will allow different software layers to interact/cooperate with each other. Specifically, the upper layers in the software stack extract high-level access pattern information and pass it to the lower layers in the stack, which in turn exploit them to achieve ultra-scalability. In particular, the main objectives of this project are: (1) Techniques, tools and software for extracting data access patterns and data-flow at runtime; (2) Interfaces and strategies for passing access pattern across the different layers for optimizations; (3) Implementation of these techniques in appropriate layers such as parallel file system, communication software 9e.g., MPI2), and runtime libraries to reduce or eliminate synchronization and locking; (4) Runtime techniques and tools that exploit access patterns for reducing power consumption and cooling requirements for the underlying storage system; and (5) Development of interfaces and software to use active storage for data analysis and filtering