Elements:Software:NSCI: Empowering Data-driven Discovery with a Provenance Collection, Management, and Analysis Software Infrastructure

元素：软件：NSCI：通过来源收集、管理和分析软件基础设施支持数据驱动的发现

• 批准号: 1835892
• 负责人: Yong Chen
• 金额: $ 60万
• 依托单位: Texas Tech University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1835892&HistoricalAwards=false
• 关键词: Elements; Software; NSCI; Empowering; Data

Scientific breakthroughs are increasingly powered by advanced computing and data analysis capabilities delivered by high performance computing (HPC) systems. In the meantime, many scientific problems have moved to a level of complexity that the ability of understanding the results, auditing how a result is generated, and reproducing the important experiments or simulation results, is critical to scientists. Enabling such a capability in HPC systems requires a holistic collection, management, and analysis software infrastructure for "provenance" data, the metadata that describes the history of a piece of data. Such a software infrastructure does not exist yet, which motivates the proposed software development of a lightweight provenance service. With such a software element, many advanced data management functionalities such as identifying the data sources, parameters, or assumptions behind a given result, auditing data history and usage, or understanding the detailed process that how different input data are transformed into outputs can be possible. Responding to the National Strategic Computing Initiative, this project will provide an attractive software infrastructure to future national HPC systems to improve the productivity of science in complex HPC simulation and analysis cycles. The project team will also recruit underrepresented students, mentor graduate and undergraduate students, integrate results into curriculum, and publish and disseminate results.The lightweight provenance service software on HPC systems will provide: 1) an always-on, background service that automatically and transparently collects and manages provenance for scientific applications, 2) captures comprehensive provenance with accurate causality to support a wide range of use cases, and 3) provides easy-to-use analysis tools for scientists to quickly explore and utilize the provenance. This project will integrate the development, education, and outreach efforts tightly together.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

高性能计算（HPC）系统所提供的先进计算和数据分析能力日益推动科学突破。与此同时，许多科学问题已经发展到一个复杂的水平，理解结果的能力，审计结果是如何产生的，并复制重要的实验或模拟结果，对科学家来说是至关重要的。在HPC系统中启用这样的功能需要针对“出处”数据（描述数据段历史的元数据）的整体收集、管理和分析软件基础设施。这样的软件基础设施还不存在，这激发了轻量级出处服务的拟议软件开发。有了这样的软件元素，许多高级数据管理功能就成为可能，例如识别给定结果背后的数据源、参数或假设，审计数据历史和使用情况，或者了解不同输入数据如何转换为输出的详细过程。响应国家战略计算计划，该项目将为未来的国家HPC系统提供有吸引力的软件基础设施，以提高复杂HPC模拟和分析周期的科学生产力。项目团队还将招募代表性不足的学生，指导研究生和本科生，将结果整合到课程中，并发布和传播结果。HPC系统上的轻量级出处服务软件将提供：1）始终在线的后台服务，自动透明地收集和管理科学应用的来源，2）捕获具有准确因果关系的全面出处，以支持广泛的用例，3）为科学家提供易于使用的分析工具，以快速探索和利用出处。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

MonSTer: An Out-of-the-Box Monitoring Tool for High Performance Computing Systems

MonSTer：用于高性能计算系统的开箱即用的监控工具

• DOI: 10.1109/cluster49012.2020.00022
• 发表时间: 2020
• 期刊: IEEE International Conference on Cluster Computing (CLUSTER'20
• 作者: Li, Jie;Ali, Ghazanfar;Nguyen, Ngan;Hass, Jon;Sill, Alan;Dang, Tommy;Chen, Yang
• 通讯作者: Chen, Yang

MIQS: metadata indexing and querying service for self-describing file formats

• DOI: 10.1145/3295500.3356146
• 发表时间: 2019-11
• 期刊: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis
• 作者: Wei Zhang;S. Byna;Houjun Tang;Brody Williams;Yong Chen

I/O characteristic discovery for storage system optimizations

• DOI: 10.1016/j.jpdc.2020.08.005
• 发表时间: 2021-02
• 期刊: J. Parallel Distributed Comput.
• 作者: Jiang Zhou;Yong Chen;Dong Dai;Zhuang Yu;Weiping Wang

Exploiting user activeness for data retention in HPC systems

利用用户活跃度来保留 HPC 系统中的数据

• DOI: 10.1145/3458817.3476201
• 发表时间: 2021
• 作者: Zhang, Wei;Byna, Suren;Sim, Hyogi;Lee, Sangkeun;Vazhkudai, Sudharshan;Chen, Yong
• 通讯作者: Chen, Yong

RadarViewer : Visualizing the dynamics of multivariate data

RadarViewer：可视化多元数据的动态

• DOI: 10.1145/3311790.3404538
• 发表时间: 2020
• 期刊: Proceedings of Practice and Experience in Advanced Research Computing Conference Series (PEARC'20
• 作者: Nguyen, Ngan;Hass, Jon;Chen, Yong;Li, Jie;Sill, Alan;Dang, Tommy
• 通讯作者: Dang, Tommy