Collaborative Research: Ultra-scalable system software and tools for data-intensive computing

该项目需要研究和开发，并解决超大规模并行机的软件和工具问题，特别是针对可扩展的I/O，存储和内存层次结构。基本前提是，要实现极端的可伸缩性，增量更改或适应传统（顺序扩展）接口和技术扩展数据访问和I/O将不会成功，因为它们是基于悲观和保守的假设并行，同步和数据共享模式。我们将开发创新技术来优化数据访问，这些技术利用对高级访问模式（“意图”）的理解，并通过运行时层使用这些信息来实现不同级别的锁定和同步的优化和减少/消除。所提出的机制将允许不同的软件层相互交互/合作。具体来说，软件栈中的上层提取高级访问模式信息，并将其传递给栈中的下层，后者反过来利用它们来实现超可扩展性。具体而言，本项目的主要目标是：（1）用于在运行时提取数据访问模式和数据流的技术、工具和软件;（2）用于在不同层之间传递访问模式以进行优化的接口和策略;（3）在适当的层中实现这些技术，例如并行文件系统、通信软件9，MPI 2）和运行时库，以减少或消除同步和锁定;（4）可利用访问模式的可编程技术和工具，以减少底层存储系统的功耗和冷却要求;以及（5）开发接口和软件，以使用主动存储进行数据分析和过滤

This project entails research and development and to address the software and tools problems fro ultra-scale parallel machines, especially targeted for scalable I/O, storage and memory hierarchy. The fundamental premise is that to achieve extreme scalability, incremental changes or adaptation of traditional (extension of sequential) interfaces and techniques for scaling data accesses and I/O will not succeed, because they are based on pessimistic and conservative assumptions of parallelism, synchronization, and data sharing patterns. We will develop innovative techniques to optimize data access that utilize the understanding of high-level access patterns ("intent"), and use that information through runtime layers to enable optimizations and reduction / elimination of locking and synchronization at different levels. The proposed mechanisms will allow different software layers to interact/cooperate with each other. Specifically, the upper layers in the software stack extract high-level access pattern information and pass it to the lower layers in the stack, which in turn exploit them to achieve ultra-scalability. In particular, the main objectives of this project are: (1) Techniques, tools and software for extracting data access patterns and data-flow at runtime; (2) Interfaces and strategies for passing access pattern across the different layers for optimizations; (3) Implementation of these techniques in appropriate layers such as parallel file system, communication software 9e.g., MPI2), and runtime libraries to reduce or eliminate synchronization and locking; (4) Runtime techniques and tools that exploit access patterns for reducing power consumption and cooling requirements for the underlying storage system; and (5) Development of interfaces and software to use active storage for data analysis and filtering