CRII: ACI: Accelerating In-Situ Scientific Data Analysis Using Software-Defined Storage Resource Enclaves
CRII:ACI:使用软件定义的存储资源飞地加速现场科学数据分析
基本信息
- 批准号:1565338
- 负责人:
- 金额:$ 17.4万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2016
- 资助国家:美国
- 起止时间:2016-06-01 至 2019-05-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
Data intensive knowledge discovery requires scientific applications to run concurrently with analytics and visualization codes, executing in situ for timely output inspection and knowledge extraction. Consequently, the Input/Output (I/O) pipelines for large scientific data analysis can be long and complex because they comprise many "stages" of analytics across different layers of the I/O stack of high-performance computing systems. Performance limitations at any I/O layer can cause an I/O bottleneck resulting in longer than expected end-to-end I/O latency. In this project, PI aims to implement a novel data management infrastructure called Software-defined Storage Resource Enclaves (SIREN) at system levels to enforce end-to-end policies that dictate an I/O pipeline's performance. The cross-cutting nature of the technologies developed in the project can help large scientific data analytics leverage the full capability of memory and storage devices on supercomputers. The project will facilitate the development of a graduate level data-intensive computing course at Washington State University Vancouver, and contribute to the education of undergraduate, female, and under-representative students. Therefore, this research aligns with the NSF mission to promote the progress of science and to advance the national prosperity and welfare.The technical objectives of the project are three-fold. First, SIREN aims to allow administrators to set allocations for enclaves to manage a group of applications that belong to the same I/O pipeline. Second, it intends to enforce I/O policies (e.g., proportional sharing) at more than one layer of I/O stacks simultaneously considering characteristics of storage devices (e.g., disparity of read/write capacity for SSDs and performance sensitivity to data locality for disks) to achieve optimal performance. Third, PI aims to solve storage-specific implementation issues, including design of user-friendly interfaces, enclave naming and resolution, metadata management, failure handling, and admission control. The introduction of SIREN can fundamentally change the execution model of data staging services widely used on supercomputers. It will also contribute to the understanding of performance characteristics of I/O pipelines under external I/O interference during data staging.
数据密集型知识发现要求科学应用程序与分析和可视化代码同时运行,并在现场执行,以及时进行输出检查和知识提取。因此,用于大型科学数据分析的输入/输出(I/O)管道可能又长又复杂,因为它们包含跨越高性能计算系统I/O堆栈的不同层的许多分析“阶段”。任何I/O层的性能限制都可能导致I/O瓶颈,从而导致比预期更长的端到端I/O延迟。在该项目中,PI的目标是在系统级别实施一种名为软件定义存储资源包围区(SIREN)的新型数据管理基础设施,以实施决定I/O管道性能的端到端策略。该项目开发的技术的交叉性质可以帮助大型科学数据分析利用超级计算机上内存和存储设备的全部能力。该项目将促进温哥华华盛顿州立大学研究生水平的数据密集型计算课程的发展,并有助于本科生、女性学生和代表性不足学生的教育。因此,这项研究与NSF促进科学进步、促进国家繁荣和福祉的使命是一致的。该项目的技术目标有三个。首先,SIREN旨在允许管理员为Enclaves设置分配,以管理属于同一I/O管道的一组应用程序。其次,它打算同时考虑存储设备的特性(例如,SSD的读/写容量差异和磁盘对数据局部性的性能敏感性),在多层I/O堆栈上强制执行I/O策略(例如,按比例共享),以实现最佳性能。第三,PI旨在解决特定于存储的实施问题,包括设计用户友好的界面、Enclave命名和解析、元数据管理、故障处理和准入控制。SIREN的引入可以从根本上改变超级计算机上广泛使用的数据暂存服务的执行模式。这也将有助于理解数据转移过程中I/O管道在外部I/O干扰下的性能特征。
项目成果
期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
数据更新时间:{{ journalArticles.updateTime }}
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Xuechen Zhang其他文献
Flexpath: Type-Based Publish/Subscribe System for Large-Scale Science Analytics
Flexpath:用于大规模科学分析的基于类型的发布/订阅系统
- DOI:
10.1109/ccgrid.2014.104 - 发表时间:
2014 - 期刊:
- 影响因子:0
- 作者:
Jai Dayal;Drew Bratcher;G. Eisenhauer;K. Schwan;M. Wolf;Xuechen Zhang;H. Abbasi;S. Klasky;N. Podhorszki - 通讯作者:
N. Podhorszki
Comparing the variations and influencing factors of CH<sub>4</sub> emissions from paddies and wetlands under CO<sub>2</sub> enrichment: A data synthesis in the last three decades
- DOI:
10.1016/j.envres.2023.115842 - 发表时间:
2023-07-01 - 期刊:
- 影响因子:
- 作者:
Haiyang Yu;Xuechen Zhang;Xiangtian Meng;Dan Luo;Zhengfu Yue;Yaying Li;Yongxiang Yu;Huaiying Yao - 通讯作者:
Huaiying Yao
Screening of molecular markers associated with hornless traits in Qira black sheep
- DOI:
10.1186/s12864-025-11608-8 - 发表时间:
2025-05-08 - 期刊:
- 影响因子:3.700
- 作者:
Wen Zhou;Lijun Zhu;Yuwei Peng;Xiaopeng Li;Xuechen Zhang;Zhipeng Han;Jingping Li;Xinyu Bai;Qifeng Gao;Ruizhi Yang;Tao Jiang;Shudong Liu - 通讯作者:
Shudong Liu
Hotspot enlargement and shortening hot moments in the rhizosphere to acquire labile phosphorus from fungal necromass in response to warming effects
热点放大和缩短根际中的热瞬态以响应变暖效应从真菌残体获取不稳定磷
- DOI:
10.1016/j.apsoil.2024.105740 - 发表时间:
2024-12-01 - 期刊:
- 影响因子:5.000
- 作者:
Duyen Thi Thu Hoang;Ali Feizi;Viola Stelmach-Kardel;Kazem Zamanian;Xuechen Zhang;Marius Schmitt;Michaela A. Dippold;Agata Gryta;Magdalena Frąc;Bahar S. Razavi - 通讯作者:
Bahar S. Razavi
Lightweight Feature Fusion Network for Single Image Super-Resolution
用于单图像超分辨率的轻量级特征融合网络
- DOI:
10.1109/lsp.2018.2890770 - 发表时间:
2019-01 - 期刊:
- 影响因子:3.9
- 作者:
Wenming Yang;Wei Wang;Xuechen Zhang;Shuifa Sun;Qingmin Liao - 通讯作者:
Qingmin Liao
Xuechen Zhang的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Xuechen Zhang', 18)}}的其他基金
Collaborative Research: REU Site: Advancing Data-Driven Deep Coupling of Computational Simulations and Experiments
合作研究:REU 站点:推进数据驱动的计算模拟和实验的深度耦合
- 批准号:
2243980 - 财政年份:2023
- 资助金额:
$ 17.4万 - 项目类别:
Standard Grant
CNS Core: Small: RUI: A Holistic Approach to Taming Unaligned Writes in Flash Drives
CNS 核心:小型:RUI:驯服闪存驱动器中未对齐写入的整体方法
- 批准号:
1906541 - 财政年份:2019
- 资助金额:
$ 17.4万 - 项目类别:
Standard Grant
相似海外基金
ACI-OPT: Aircraft Component Installation Optimization
ACI-OPT:飞机部件安装优化
- 批准号:
576486-2022 - 财政年份:2022
- 资助金额:
$ 17.4万 - 项目类别:
Alliance Grants
Aircraft Measurements of Volcanic Aerosol-Cloud Interactions (Vol-ACI)
火山气溶胶-云相互作用的飞机测量 (Vol-ACI)
- 批准号:
NE/W005018/1 - 财政年份:2021
- 资助金额:
$ 17.4万 - 项目类别:
Research Grant
CRII: ACI: Unveiling the Origin of the Highest Energy Particles in the Universe with Large-Scale First-Principle Fully-Kinetic Simulations
CRII:ACI:通过大规模第一原理全动力学模拟揭示宇宙中最高能量粒子的起源
- 批准号:
1657507 - 财政年份:2017
- 资助金额:
$ 17.4万 - 项目类别:
Standard Grant
Collaborative Research: ACI-CDS&E: Highly Parallel Algorithms and Architectures for Convex Optimization for Realtime Embedded Systems (CORES)
合作研究:ACI-CDS
- 批准号:
1708299 - 财政年份:2017
- 资助金额:
$ 17.4万 - 项目类别:
Standard Grant
Collaborative Research: ACI-CDS&E: Highly Parallel Algorithms and Architectures for Convex Optimization for Realtime Embedded Systems (CORES)
合作研究:ACI-CDS
- 批准号:
1709069 - 财政年份:2017
- 资助金额:
$ 17.4万 - 项目类别:
Standard Grant
CRII: ACI: 4D Dynamic Anisotropic Meshing and Applications
CRII:ACI:4D 动态各向异性网格划分和应用
- 批准号:
1657364 - 财政年份:2017
- 资助金额:
$ 17.4万 - 项目类别:
Standard Grant
CRII: SHF: ACI: Performance-in-Depth Sparse Solvers for Heterogeneous Parallel Platforms.
CRII:SHF:ACI:异构并行平台的深度性能稀疏求解器。
- 批准号:
1657175 - 财政年份:2017
- 资助金额:
$ 17.4万 - 项目类别:
Standard Grant
CRII: ACI: Algorithms and Tools to Facilitate the Development of High Fidelity Reactive Molecular Dynamics Models
CRII:ACI:促进高保真反应分子动力学模型开发的算法和工具
- 批准号:
1566049 - 财政年份:2016
- 资助金额:
$ 17.4万 - 项目类别:
Standard Grant
Mobiliser les expertises, consolider les acquis et agir pour promouvoir les saines habitudes de vie dans le système de santé : une approche d'application des connaissances intégrée (ACi) au Québec
调动专业知识,巩固并促进在健康系统中生活的习惯:魁北克省综合认知应用程序 (ACi)
- 批准号:
343617 - 财政年份:2016
- 资助金额:
$ 17.4万 - 项目类别:
Miscellaneous Programs
CRII: ACI: Efficient Radiative Heat Transfer Modeling In Large-Scale Combustion Systems
CRII:ACI:大型燃烧系统中的高效辐射传热建模
- 批准号:
1566259 - 财政年份:2016
- 资助金额:
$ 17.4万 - 项目类别:
Standard Grant