Collaborative Research: SI2-SSI: Cyberinfrastructure for Advancing Hydrologic Knowledge through Collaborative Integration of Data Science, Modeling and Analysis

合作研究：SI2-SSI：通过数据科学、建模和分析的协作集成推进水文知识的网络基础设施

• 批准号: 1664119
• 负责人: Shaowen Wang
• 金额: $ 70万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-10-01 至 2022-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1664119&HistoricalAwards=false
• 关键词: Collaborative; Research; SI2; SSI; Cyberinfrastructure

Researchers across the country and around the world expend tremendous resources to gather and analyze vast stores of hydrologic data and populate a myriad of models to better understand hydrologic phenomena and find solutions to vexing water problems. Each of those researchers has limited money, time, computational capacity, data storage, and ability to put that data to productive use. What if they could combine their efforts to make collaboration easier? What if those collected data sets and processed model outputs could be used collaboratively to help advance hydrologic understanding beyond their original purpose? HydroShare is a system to advance hydrologic science by enabling the scientific community to more easily and freely share products resulting from their research, not just the scientific publication summarizing a study, but also the data and models used to create the scientific publication. HydroShare supports the sharing and publication of hydrologic data and models. This capability is necessary for community model development, execution, and evaluation and to improve reproducibility and community trust in scientific findings through transparency. As a platform for collaboration and running models on advanced computational infrastructure, HydroShare enhances the capability for data intensive research in hydrology and other aligned sciences. HydroShare is designed to help researchers easily meet the sharing requirements of data management plans while at the same time providing value added functionality that makes metadata capture more effective and helps researchers improve their work productivity. This project will extend the capabilities of the HydroShare cyberinfrastructure to enhance support for scientific methods, advance the social capabilities of HydroShare to enable improved collaborative research, integrate with 3rd party consumer data storage systems to provide more flexible and sustainable data storage. and establish an application testing environment to empower researchers to develop their own computer programs to act on and work with data in HydroShare. Empowering HydroShare users with the ability to rapidly develop web application programs opens the door to unforeseen, innovative combinations of data and models. WRF-Hydro, the framework for the NOAA National Water Model, will be used as a use case for collaboration on model development. Since WRF-Hydro is used by NOAA as part of the National Water Model (NWM), this collaboration opens possibilities for transfer of research to operations. Collectively, this functionality will provide a computing framework for transforming the practice of broad science communities to leverage advances in data science and computation and accelerate discovery.HydroShare is a system for sharing hydrologic data and models aimed at giving hydrologists the cyberinfrastructure needed to manage data, innovate and collaborate in research to solve water problems. It addresses the challenges of sharing data and hydrologic models to support collaboration and reproducible hydrologic science through the publication of hydrologic data and models. With HydroShare users can: (1) share data and models with colleagues; (2) manage who has access to shared content; (3) share, access, visualize and manipulate a broad set of hydrologic data types and models; (4) use the web services interface to program automated and client access; (5) publish data and models to meet the requirements of research project data management plans; (6) discover and access data and models published by others; and (7) use web apps to visualize, analyze, and run models on data. This project will extend the capabilities of HydroShare to: (1) enhance support for scientific methods enabling systematic data and model analysis and hypothesis testing; (2) advance the social capabilities of HydroShare to enable improved collaborative research; (3) integrate with 3rd party consumer data storage systems to provide more flexible and sustainable data storage; and (4) establish an application testing environment to empower researchers to develop their own computer programs to act on and work with data in HydroShare. Under development since 2012 and first released in 2014, HydroShare supports the sharing and publication of hydrologic data and models. This capability is necessary for community model development, execution, and evaluation. As a platform for collaboration and cloud based computation on network servers remote from the user, HydroShare enhances the capability for data intensive research in hydrology and other aligned sciences. HydroShare is innovative from a computer science and CI perspective in the way computation and data sharing are framed as a network computing platform that integrates data storage, organization, discovery, and programmable actions through web applications (web apps). Support for these three key elements of computation allows researchers to easily employ services beyond the desktop to make data storage and manipulation more reliable and scalable, while improving ability to collaborate and reproduce results. The generation of new understanding, through integration of information from multiple sources and reuse and collaborative enrichment of research data and models, will be enhanced. Structured and systematic model process intercomparisons and alternative hypothesis testing will be enabled, bringing, through user friendly CI, the latest thinking in advancing hydrologic modeling to a broad community of earth science researchers, thereby transforming research practices and the knowledge generated from this research. Interoperability with consumer cloud storage will greatly ease entry of content into HydroShare and support its sustainability. This meshing of the rigorous metadata model of HydroShare with consumer file sharing will enhance reproducibility as well as provide an innovative mechanism for sharing and collaboration. Empowering HydroShare users with the ability to rapidly develop web apps opens the door to unforeseen, innovative combinations of data and models. WRF- Hydro will be used as a use case for collaboration on model development. WRF-Hydro provides a reach-based high resolution representation of hydrologic processes, and offers the potential to bring together scientists working at scales from research catchments on the order of 1 to 100s of square kilometers as well as those working at regional to continental scales and cut across disciplines from environmental engineering to aquatic ecologists. Since WRF-Hydro is used by NOAA as part of the National Water Model (NWM), this collaboration opens possibilities for transfer of research to operations. This project will adapt current best practices in CI for interoperability and extensibility to serve this multidisciplinary community of scientists. HydroShare has already had a broader impact, with documented rapid growth in use and uptake by other projects including in EarthCube. It will become sustainable community CI through operation as part of the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) Water Data Center (WDC) facility. The use of WRF- Hydro/NWM, as a driving use case, will advance CI for community based model improvement. Through the Summer Young Innovators Program at the National Water Center (NWC), supported by the National Weather Service (NWS) and operated by CUAHSI, a pathway already exists to translate research findings to the operational needs of federal agencies participating in the NWC. HydroShare already touches a broad and diverse community, with user base including Native American tribes, hydrologic science students, and faculty researchers across the U.S. This proposal builds on the success of HydroShare to extend its capabilities and broaden model hypothesis testing, collaborative data sharing, and open app development across earth science research and education.

全国和世界各地的研究人员花费了大量资源来收集和分析大量的水文数据，并填充无数的模型，以更好地了解水文现象并找到解决棘手水问题的解决方案。每个研究人员的资金、时间、计算能力、数据存储以及将数据用于生产性用途的能力都是有限的。如果他们能够共同努力让协作变得更加容易，结果会怎样呢？如果这些收集到的数据集和处理后的模型输出可以协同使用，以帮助推进水文理解超出其最初目的，结果会怎样呢？ HydroShare 是一个推进水文科学的系统，它使科学界能够更轻松、更自由地共享其研究成果，不仅是总结研究的科学出版物，还包括用于创建科学出版物的数据和模型。 HydroShare 支持水文数据和模型的共享和发布。这种能力对于社区模型的开发、执行和评估以及通过透明度提高科学发现的可重复性和社区信任是必要的。作为在先进计算基础设施上进行协作和运行模型的平台，HydroShare 增强了水文学和其他相关科学领域的数据密集型研究的能力。 HydroShare 旨在帮助研究人员轻松满足数据管理计划的共享要求，同时提供增值功能，使元数据捕获更加有效，并帮助研究人员提高工作效率。该项目将扩展 HydroShare 网络基础设施的功能，以增强对科学方法的支持，提高 HydroShare 的社交能力，以改善协作研究，与第三方消费者数据存储系统集成，以提供更灵活和可持续的数据存储。建立应用程序测试环境，使研究人员能够开发自己的计算机程序，以对 HydroShare 中的数据进行操作和处理。让 HydroShare 用户能够快速开发 Web 应用程序，为不可预见的创新数据和模型组合打开了大门。 WRF-Hydro 是 NOAA 国家水模型的框架，将用作模型开发合作的用例。由于 WRF-Hydro 被 NOAA 用作国家水模型 (NWM) 的一部分，因此这种合作为将研究转移到运营提供了可能性。总的来说，该功能将为广大科学界的实践提供一个计算框架，以利用数据科学和计算的进步并加速发现。HydroShare 是一个共享水文数据和模型的系统，旨在为水文学家提供管理数据、创新和协作研究以解决水问题所需的网络基础设施。它解决了共享数据和水文模型的挑战，通过发布水文数据和模型来支持协作和可重复的水文科学。通过 HydroShare，用户可以：（1）与同事共享数据和模型； (2) 管理谁有权访问共享内容； (3) 共享、访问、可视化和操作广泛的水文数据类型和模型； (4) 使用Web服务接口进行自动化编程和客户端访问； （五）发布符合科研项目数据管理计划要求的数据和模型； （6）发现和获取他人发布的数据和模型； (7) 使用网络应用程序对数据进行可视化、分析和运行模型。该项目将扩展 HydroShare 的功能：(1) 加强对科学方法的支持，实现系统数据和模型分析以及假设检验； (2) 提高 HydroShare 的社交能力，以改善协作研究； （3）与第三方消费者数据存储系统集成，提供更灵活、可持续的数据存储； (4) 建立应用程序测试环境，使研究人员能够开发自己的计算机程序，以对 HydroShare 中的数据进行操作和处理。 HydroShare 自 2012 年开发并于 2014 年首次发布，支持水文数据和模型的共享和发布。此功能对于社区模型开发、执行和评估是必需的。作为远离用户的网络服务器上的协作和基于云计算的平台，HydroShare 增强了水文学和其他相关科学领域的数据密集型研究的能力。从计算机科学和 CI 的角度来看，HydroShare 的创新在于将计算和数据共享构建为一个网络计算平台，通过 Web 应用程序（Web 应用程序）集成数据存储、组织、发现和可编程操作。对这三个关键计算要素的支持使研究人员能够轻松使用桌面之外的服务，使数据存储和操作更加可靠和可扩展，同时提高协作和重现结果的能力。通过整合多个来源的信息以及研究数据和模型的重用和协作丰富，将增强新理解的产生。将启用结构化和系统化的模型过程比较和替代假设检验，通过用户友好的 CI，将推进水文建模的最新思想带给广泛的地球科学研究人员，从而改变研究实践和从研究中产生的知识。与消费者云存储的互操作性将大大简化 HydroShare 的内容输入并支持其可持续性。 HydroShare 严格的元数据模型与消费者文件共享的结合将提高可重复性，并提供创新的共享和协作机制。让 HydroShare 用户能够快速开发 Web 应用程序，为不可预见的数据和模型的创新组合打开了大门。 WRF-Hydro 将用作模型开发协作的用例。 WRF-Hydro 提供了基于河段的水文过程高分辨率表示，并提供了将 1 至 100 平方公里研究流域规模的科学家以及区域到大陆规模的科学家聚集在一起的潜力，以及从环境工程到水生生态学家等跨学科的科学家。由于 WRF-Hydro 被 NOAA 用作国家水模型 (NWM) 的一部分，因此这种合作为将研究转移到运营提供了可能性。该项目将采用 CI 中当前的最佳实践来实现互操作性和可扩展性，以服务于这个多学科科学家社区。 HydroShare 已经产生了更广泛的影响，据记录，包括 EarthCube 在内的其他项目的使用和吸收量迅速增长。通过作为水文科学促进大学联盟 (CUAHSI) 水数据中心 (WDC) 设施的一部分，它将成为可持续社区 CI。使用 WRF-Hydro/NWM 作为驾驶用例，将促进基于社区的模型改进的 CI。通过由国家气象局 (NWS) 支持并由 CUAHSI 运营的国家水中心 (NWC) 夏季青年创新者计划，已经存在一条将研究成果转化为参与 NWC 的联邦机构的业务需求的途径。 HydroShare 已经触及广泛而多样化的社区，用户群包括美国原住民部落、水文科学学生和美国各地的教师研究人员。该提案以 HydroShare 的成功为基础，扩展了其功能，并扩大了模型假设检验、协作数据共享以及跨地球科学研究和教育的开放应用程序开发。

项目成果

OpenCLC: An open-source software tool for similarity assessment of linear hydrographic features

OpenCLC：用于线性水文特征相似性评估的开源软件工具

• DOI: 10.1016/j.softx.2020.100401
• 发表时间: 2020
• 期刊: SoftwareX
• 影响因子: 3.4
• 作者: Li, Ting;Stanislawski, Lawrence V.;Brockmeyer, Tyler;Wang, Shaowen;Shavers, Ethan
• 通讯作者: Shavers, Ethan

Reproducible Hydrological Modeling with CyberGIS-Jupyter: A Case Study on SUMMA

使用 Cyber​​GIS-Jupyter 进行可重复水文建模：SUMMA 案例研究

• DOI: 10.1145/3332186.3333052
• 发表时间: 2019
• 期刊: Proceedings of the Practice and Experience in Advanced Research Computing on Rise of the Machines (learning
• 作者: Lyu, Fangzheng;Yin, Dandong;Padmanabhan, Anand;Choi, Youngdon;Goodall, Jonathan L.;Castronova, Anthony;Tarboton, David;Wang, Shaowen
• 通讯作者: Wang, Shaowen

Enabling Collaborative Numerical Modeling in Earth Sciences using Knowledge Infrastructure

• DOI: 10.1016/j.envsoft.2019.03.020
• 发表时间: 2019-10-01
• 期刊: ENVIRONMENTAL MODELLING & SOFTWARE
• 影响因子: 4.9
• 作者: Bandaragoda, C.;Castronova, A.;Wang, S.
• 通讯作者: Wang, S.

CyberGIS‐Jupyter for reproducible and scalable geospatial analytics

Cyber​​GIS – Jupyter 用于可重复和可扩展的地理空间分析

• DOI: 10.1002/cpe.5040
• 发表时间: 2018
• 期刊: Concurrency and Computation: Practice and Experience
• 作者: Yin, Dandong;Liu, Yan;Hu, Hao;Terstriep, Jeff;Hong, Xingchen;Padmanabhan, Anand;Wang, Shaowen
• 通讯作者: Wang, Shaowen

Bridging Local Cyberinfrastructure and XSEDE with CyberGIS-Jupyter

使用 Cyber​​GIS-Jupyter 连接本地网络基础设施和 XSEDE

• DOI: 10.1145/3332186.3333257
• 发表时间: 2019
• 期刊: Practice and Experience in Advanced Research Computing (PEARC19
• 作者: Padmanabhan, Anand;Yin, Dandong;Lyu, Fangzheng;Wang, Shaowen
• 通讯作者: Wang, Shaowen