EAGER: Dynamic Data Path Management for Asynchronous Vertical Storage Hierarchy

人类对世界的了解以及通过科学、技术、工程和数学（STEM）来改善生活质量和理解的能力越来越依赖于存储、访问和管理代表科学和过程测量、科学和工程模拟结果以及长期知识的超大型持久数据集的能力。超级计算机通常以双重或独立的模式运行：一种模式在其临时（短暂）主存储器中进行计算，另一种模式用于监督大型持久数据存储器的使用。随着超级计算机变得越来越大，到本世纪末可能达到Exaflops的规模，大容量存储的可比规模和易用性受到严重挑战。这项研究将通过垂直内存层次结构解决数据迁移的效率和可扩展性问题，并将统一主存数据对象和持久存储数据的命名方式，创建一个单一的，易于使用的编程。这将彻底改变数据密集型超级计算，并为未来的Exascale系统设计和编程开辟新的道路。这项研究是与克莱姆森大学合作，提供一个概念验证系统来评估新的概念。在主存储器中的计算和持久数据的大容量存储器的操作之间的语义和性能障碍对性能和可编程性施加了重大限制。由于访问延迟时间的不确定性与开销相结合，并且需要利用数据访问并行性来实现高吞吐量，因此需要在临时存储和持久对象之间建立一种新的关系，以统一它们的关联并管理操作的重复性，同时实现高效率。这项研究正在推导一个创新的执行模型，并开发一个概念验证实验系统，以测试和评估其潜在的概念，为新一代的持久性大容量存储在极端规模。它将解决的挑战，并提供了一种手段，通过一个单一的抽象的数据操作和元数据和同步的整合，以管理响应时间的不确定性和逻辑冲突的访问，同时自动隐藏延迟的语义统一的短暂和大容量存储。新模型将支持异步垂直存储层次结构的动态数据路径管理，利用自适应运行时事件驱动技术来提高效率和可扩展性，包括数据垂直传输的管理，这需要一个动态控制整个数据路径的创新策略。

Humankind's knowledge of the world and its ability to manipulate it for the betterment of quality of life and understanding through science, technology, engineering, and mathematics (STEM) is increasingly dependent on the ability to store, access, and manage extremely large persistent data sets representing scientific and process measurements, results from science and engineering simulations, and long-term knowledge. Supercomputers conventionally operate in dual or separate modes: one to do the computations in their temporary (ephemeral)-main memory-and the other to supervise the use of large persistent data storage. As supercomputers get larger, perhaps to the scale of an Exaflops by the end of this decade, the comparable scale and ease of use of mass storage is severely challenged. This research will address the problems of efficiency and scalability of data migration through the vertical memory hierarchy and will unify the way both main memory data objects and persistent storage data are named creating a single, easy to use programming. This will revolutionize data intensive supercomputing and establish a new path towards future Exascale system design and programming. This research is in collaboration with Clemson University to provide a proof-of-concept system to evaluate the new concepts.The semantic and performance barriers between computing in main memory and manipulation of mass storage for persistent data have imposed significant limitations to performance and programmability. Because of uncertainties of access latency times combined with overheads and the need to exploit data access parallelism for high throughput, a new relationship between ephemeral storage and persistent objects is needed to unify their association and manage the asynchrony of operation while achieving high efficiency. This research is deriving an innovative execution model and developing a proof-of-concept experimental system to test and evaluate its underlying concepts for a new generation of persistent mass storage at extreme scale. It will address the challenges and provide the means for the unification of the semantics of ephemeral and mass storage through a single abstraction of data manipulation and the integration of meta-data and synchronization to manage asynchrony and uncertainty of response time as well as logical conflicting accesses while automatically hiding latency. The new model will support dynamic data path management for the asynchronous vertical storage hierarchy, exploiting adaptive runtime event-driven techniques for enhanced efficiency and scalability including management of vertical transport of data, which demands an innovative strategy of dynamic control of the entire data path.