Collaborative Proposal: Ultra-scalable system software and tools for data-intensive computing

该项目需要研究和开发，并解决超大规模并行机器的软件和工具问题，特别是针对可扩展的I/O，存储和内存层次结构。基本前提是，要实现极端的可伸缩性，增量更改或适应传统的（顺序扩展）接口和技术来扩展数据访问和I/O将不会成功，因为它们是基于对并行性、同步和数据共享模式的悲观和保守假设。我们将开发创新的技术来优化数据访问，利用对高级访问模式（“意图”）的理解，并通过运行时层使用该信息来实现不同级别上的优化和减少/消除锁定和同步。所提出的机制将允许不同的软件层相互交互/合作。具体来说，软件堆栈中的上层提取高级访问模式信息并将其传递给堆栈中的下层，而下层又利用这些信息来实现超可伸缩性。具体来说，该项目的主要目标是：(1)用于在运行时提取数据访问模式和数据流的技术、工具和软件；(2)跨层传递访问模式的接口和策略进行优化；(3)在适当的层（如并行文件系统、通信软件等）实现这些技术。（MPI2）和运行时库，以减少或消除同步和锁定；(4)利用访问模式来降低底层存储系统的功耗和冷却要求的运行时技术和工具；(5)开发使用主动存储进行数据分析和过滤的接口和软件

This project entails research and development and to address the software and tools problems fro ultra-scale parallel machines, especially targeted for scalable I/O, storage and memory hierarchy. The fundamental premise is that to achieve extreme scalability, incremental changes or adaptation of traditional (extension of sequential) interfaces and techniques for scaling data accesses and I/O will not succeed, because they are based on pessimistic and conservative assumptions of parallelism, synchronization, and data sharing patterns. We will develop innovative techniques to optimize data access that utilize the understanding of high-level access patterns ("intent"), and use that information through runtime layers to enable optimizations and reduction / elimination of locking and synchronization at different levels. The proposed mechanisms will allow different software layers to interact/cooperate with each other. Specifically, the upper layers in the software stack extract high-level access pattern information and pass it to the lower layers in the stack, which in turn exploit them to achieve ultra-scalability. In particular, the main objectives of this project are: (1) Techniques, tools and software for extracting data access patterns and data-flow at runtime; (2) Interfaces and strategies for passing access pattern across the different layers for optimizations; (3) Implementation of these techniques in appropriate layers such as parallel file system, communication software 9e.g., MPI2), and runtime libraries to reduce or eliminate synchronization and locking; (4) Runtime techniques and tools that exploit access patterns for reducing power consumption and cooling requirements for the underlying storage system; and (5) Development of interfaces and software to use active storage for data analysis and filtering