CRII: CSR: Enhancing Eventual Data Consistency in Multidimensional Scientific Computing through Lightweight In-Memory Distributed Ledger System.

点击翻译按钮获取中文摘要

In the realm of scientific computing, ensuring the consistency of data over time is essential for facilitating collaboration and advancing research, especially when dealing with complex multidimensional data. Numerous scientific endeavors involve interconnected components exchanging critical information, such as climate modeling simulations and DNA sequencing. The ability to maintain eventual consistency allows individual components to operate independently (achieving improved efficiency), yet ensures their data updates eventually align, even amidst potential disruptions. However, achieving this level of consistency when managing vast volumes of data poses significant challenges, including higher latencies, complex data synchronization, and scalability issues. Current solutions are inadequate in providing lightweight in-memory eventual consistency for large-scale systems. This project aims to bridge this gap by crafting a robust eventual consistency model tailored for extreme-scale systems. The proposed model comprises two core modules -- one dedicated to refining data modeling strategies to reduce complexity and optimize workload distribution, and the other aimed at establishing a scalable in-memory distributed ledger for enhanced caching and processing efficiency.The proposed research promises to improve data management practices, particularly for intricate datasets, thereby fostering advancements in scientific research. These innovations stand to benefit the scientific community by introducing advancements in the management of large, complex datasets within extreme-scale systems. Through the integration of lightweight eventual consistency into these systems, the project envisions a substantial impact on data handling techniques. This interdisciplinary initiative, spanning distributed databases, systems, and high-performance computing, will foster cross-disciplinary innovation across diverse scientific domains. Further, the resulting systems derived from this research will play a role in advancing scientific research and enabling more efficient collaboration among researchers and scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.