Collaborative Research: CNS core: OAC core: Small: New Techniques for I/O Behavior Modeling and Persistent Storage Device Configuration

合作研究： CNS 核心：OAC 核心：小型：I/O 行为建模和持久存储设备配置新技术

• 批准号: 2008072
• 负责人: Ningfang Mi
• 金额: $ 24.49万
• 依托单位: Northeastern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-05-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008072&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; core; OAC

Currently, there is a rapidly growing diversity in data processing workloads. Likewise, new advancements in persistent storage technologies are emerging. Therefore, it is important to have new techniques for benchmarking and appropriately configuring storage systems in order to obtain the best possible performance and reliability. This project proposes to derive new input/output (I/O) models to capture I/O behaviors accurately when running multiple applications with different workloads on storage systems such as flash-based solid-state drives (SSDs). In addition, this project develops new approaches to identify the most appropriate internal algorithm for different types of persistent storage devices and dynamically adjust the associated algorithm parameters according to I/O activities.This project makes empirical contributions to storage systems by addressing challenges issued by large-scale data-intensive applications. Specifically, it advances (1) how to analyze the impact of various system components while running multiple workloads on emerging storage systems; (2) how to design interactive frameworks that allow users to modify the internal algorithms and parameters of modern storage devices; (3) how to enable novices to configure storage systems with respect to their workloads and data processing requirements; and (4) how to derive I/O models to predict future I/O workload patterns and accordingly configure storage systems in advance for better performance.This project will lead to better storage systems design with high performance and reliability. The outcome of this project will bring a significant impact on many areas that are dependent on processing a large amount of data. This project will share the findings with undergraduate and graduate students through computer science and engineering programs and open up career opportunities to female students, underrepresented minorities, and first-generation college students. This project will disseminate the proposed techniques into the industry and foster technology transfer through new industrial collaborations. The developed infrastructure will be available to the research community through a web-based portal.All the publicly disclosable NSF funded work products developed under this project will be maintained at the project website (https://damrl.cis.fiu.edu/research/) at Florida International University (FIU) for at least five years beyond the end of the project. Data generated and collected as part of this project will be deposited into Digital Repository Service (DRS) (https://repository.library.northeastern.edu/) at Northeastern University (NEU) and maintained for at least 5 years beyond the end of the project. The developed software code and tools will be published in scholarly articles and be made available online via NEU's DRS, and FIU's project website.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当前，数据处理工作负载的多样性正在迅速增长。同样，持久存储技术的新进展也在不断涌现。因此，为了获得最佳的性能和可靠性，有必要采用新的技术对存储系统进行基准测试和适当配置。本项目提出推导新的输入/输出（I/O）模型，以便在存储系统（如基于闪存的固态驱动器（ssd））上运行具有不同工作负载的多个应用程序时准确捕获I/O行为。此外，该项目还开发了新的方法，用于为不同类型的持久存储设备确定最合适的内部算法，并根据I/O活动动态调整相关的算法参数。该项目通过解决大规模数据密集型应用程序所带来的挑战，为存储系统做出了经验贡献。具体来说，它提出了(1)如何在新兴存储系统上运行多个工作负载时分析各种系统组件的影响；(2)如何设计允许用户修改现代存储设备内部算法和参数的交互框架；(3)如何使新手能够根据自己的工作负荷和数据处理需求配置存储系统；(4)如何推导I/O模型来预测未来的I/O工作负载模式，并相应地提前配置存储系统以获得更好的性能。通过本课题的研究，可以更好地设计出高性能、高可靠性的存储系统。该项目的结果将对许多依赖处理大量数据的领域产生重大影响。该项目将通过计算机科学和工程项目与本科生和研究生分享研究结果，并为女学生、未被充分代表的少数民族和第一代大学生提供就业机会。该项目将向工业界传播拟议的技术，并通过新的工业合作促进技术转让。开发的基础设施将通过一个基于网络的门户提供给研究界。所有在本项目下开发的公开披露的NSF资助的工作产品将在佛罗里达国际大学（FIU）的项目网站（https://damrl.cis.fiu.edu/research/）上保存至少五年，直至项目结束。作为该项目的一部分，生成和收集的数据将存入东北大学（NEU）的数字存储库服务（DRS） （https://repository.library.northeastern.edu/），并在项目结束后至少保存5年。开发的软件代码和工具将发表在学术文章中，并通过新大学的DRS和国际金融大学的项目网站在线提供。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Performance and Consistency Analysis for Distributed Deep Learning Applications

• DOI: 10.1109/ipccc50635.2020.9391566
• 发表时间: 2020-11
• 期刊: 2020 IEEE 39th International Performance Computing and Communications Conference (IPCCC)
• 作者: Danlin Jia;M. Saha;J. Bhimani;N. Mi

DIRS: Dynamic Initial Rate Setting in Congestion Control for Disaggregated Storage Systems

• DOI: 10.1109/icnc57223.2023.10074382
• 发表时间: 2023-02
• 期刊: 2023 International Conference on Computing, Networking and Communications (ICNC)
• 作者: Xiaoqian Zhang;Allen Yang;Danlin Jia;Li Wang;Mahsa Bayati;Pradeep Subedi;Xuebin Yao;B. Sheng;N. Mi

SRC: Mitigate I/O Throughput Degradation in Network Congestion Control of Disaggregated Storage Systems

• DOI: 10.1109/ipdps54959.2023.00035
• 发表时间: 2023-05
• 期刊: 2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
• 作者: Danlin Jia;Yiming Xie;Li Wang;Xiaoqian Zhang;Allen Yang;Xuebin Yao;Mahsa Bayati;Pradeep Subedi;B. Sheng;N. Mi

Learning-Based Dynamic Memory Allocation Schemes for Apache Spark Data Processing

• DOI: 10.1109/tcc.2023.3329129
• 发表时间: 2024-01
• 期刊: IEEE Transactions on Cloud Computing
• 影响因子: 6.5
• 作者: Danlin Jia;Li Wang;Natalia Valencia;J. Bhimani;Bo Sheng;N. Mi

A Data-Loader Tunable Knob to Shorten GPU Idleness for Distributed Deep Learning

用于缩短分布式深度学习 GPU 空闲时间的数据加载器可调旋钮

• DOI: 10.1109/cloud55607.2022.00068
• 发表时间: 2022
• 期刊: 2022 IEEE 15th International Conference on Cloud Computing (CLOUD
• 作者: Jia, Danlin;Yuan, Geng;Lin, Xue;Mi, Ningfang
• 通讯作者: Mi, Ningfang