EAGER: Dynamic Data Path Management for Asynchronous Vertical Storage Hierarchy

人类对世界的认识及其通过科学、技术、工程和数学(STEM)改善生活质量和理解世界的能力越来越依赖于存储、访问和管理代表科学和过程测量、科学和工程模拟结果以及长期知识的超大持久数据集的能力。传统上，超级计算机以双模式或独立模式运行：一种是在其临时(临时)主存储器中进行计算，另一种是监督大容量永久数据存储的使用。随着超级计算机变得越来越大，到本世纪末可能会达到Exaflop的规模，大容量存储的规模和易用性受到了严峻的挑战。这项研究将解决通过垂直内存层次结构进行数据迁移的效率和可扩展性问题，并将主内存数据对象和持久存储数据的命名方式统一起来，创建一个单一的、易于使用的编程。这将使数据密集型超级计算发生革命性变化，并为未来的亿级系统设计和编程开辟一条新的道路。这项研究与克莱姆森大学合作，提供了一个概念验证系统来评估新概念。在主存中计算和处理海量存储之间的语义和性能障碍对性能和可编程性造成了很大的限制。由于访问延迟时间和开销的不确定性，以及需要利用数据访问并行性来获得高吞吐量，需要在临时存储和持久对象之间建立一种新的关系来统一它们的关联，并在获得高效率的同时管理操作的异步性。这项研究正在推导一种创新的执行模型，并开发一个概念验证实验系统，以测试和评估其在极端规模下的新一代持久海量存储的基本概念。它将解决这些挑战，并通过对数据操作的单一抽象以及元数据和同步的集成来管理响应时间的异步性和不确定性以及逻辑冲突的访问，同时自动隐藏延迟，从而提供统一临时存储和大容量存储的语义的手段。新模型将支持异步垂直存储层次结构的动态数据路径管理，利用自适应运行时事件驱动技术来提高效率和可扩展性，包括管理数据的垂直传输，这需要一种创新的策略来动态控制整个数据路径。

Humankind's knowledge of the world and its ability to manipulate it for the betterment of quality of life and understanding through science, technology, engineering, and mathematics (STEM) is increasingly dependent on the ability to store, access, and manage extremely large persistent data sets representing scientific and process measurements, results from science and engineering simulations, and long-term knowledge. Supercomputers conventionally operate in dual or separate modes: one to do the computations in their temporary (ephemeral)-main memory-and the other to supervise the use of large persistent data storage. As supercomputers get larger, perhaps to the scale of an Exaflops by the end of this decade, the comparable scale and ease of use of mass storage is severely challenged. This research will address the problems of efficiency and scalability of data migration through the vertical memory hierarchy and will unify the way both main memory data objects and persistent storage data are named creating a single, easy to use programming. This will revolutionize data intensive supercomputing and establish a new path towards future Exascale system design and programming. This research is in collaboration with Clemson University to provide a proof-of-concept system to evaluate the new concepts.The semantic and performance barriers between computing in main memory and manipulation of mass storage for persistent data have imposed significant limitations to performance and programmability. Because of uncertainties of access latency times combined with overheads and the need to exploit data access parallelism for high throughput, a new relationship between ephemeral storage and persistent objects is needed to unify their association and manage the asynchrony of operation while achieving high efficiency. This research is deriving an innovative execution model and developing a proof-of-concept experimental system to test and evaluate its underlying concepts for a new generation of persistent mass storage at extreme scale. It will address the challenges and provide the means for the unification of the semantics of ephemeral and mass storage through a single abstraction of data manipulation and the integration of meta-data and synchronization to manage asynchrony and uncertainty of response time as well as logical conflicting accesses while automatically hiding latency. The new model will support dynamic data path management for the asynchronous vertical storage hierarchy, exploiting adaptive runtime event-driven techniques for enhanced efficiency and scalability including management of vertical transport of data, which demands an innovative strategy of dynamic control of the entire data path.