Geoweaver: Advanced Cyberinfrastructure for Managing Hybrid Geoscientific AI Workflows
Geoweaver: Advanced Cyberinfrastructure for Managing Hybrid Geoscientific AI Workflows
复制标题
DOI：
10.3390/ijgi9020119
复制
发表时间：
2020-02-01
影响因子：
3.4
通讯作者：
Magill, Andrew B.
中科院分区：
文献类型：
--
作者：
Sun, Ziheng;Di, Liping;Magill, Andrew B.
基于AI（人工智能）的地理空间数据分析已经获得了很多关注。地理空间数据集是多维的;具有时空背景;以不同的格式存在;并且需要复杂的AI工作流程，不仅包括AI算法训练和测试，还包括数据预处理和结果后处理。这种复杂性对全栈人工智能工作流管理提出了巨大的挑战，因为研究人员经常使用各种时间密集型手动操作来管理他们的项目。然而，现有的工作流管理软件都没有提供一个令人满意的解决方案，混合资源，完整的文件访问，数据流，代码控制和出处。为了提高全栈人工智能工作流管理的效率，提出了一种新的工作流管理系统Geoweaver。它支持将所有预处理、人工智能训练和测试以及后处理步骤链接到一个自动化工作流程中。为了证明其实用性，我们提出了一个用例，其中Geowaver使用Landsat数据管理端到端深度学习以进行实时作物映射。我们展示了Geoweaver如何有效地消除管理各种脚本、代码、库、笔记本、数据集、服务器和平台的繁琐工作，大大减少了研究人员必须花费在这种基于AI的工作流上的时间、成本和精力。通过Geoweaver展示的概念是未来人工智能研究网络基础设施的重要组成部分。
AI (artificial intelligence)-based analysis of geospatial data has gained a lot of attention. Geospatial datasets are multi-dimensional; have spatiotemporal context; exist in disparate formats; and require sophisticated AI workflows that include not only the AI algorithm training and testing, but also data preprocessing and result post-processing. This complexity poses a huge challenge when it comes to full-stack AI workflow management, as researchers often use an assortment of time-intensive manual operations to manage their projects. However, none of the existing workflow management software provides a satisfying solution on hybrid resources, full file access, data flow, code control, and provenance. This paper introduces a new system named Geoweaver to improve the e fficiency of full-stack AI workflow management. It supports linking all the preprocessing, AI training and testing, and post-processing steps into a single automated workflow. To demonstrate its utility, we present a use case in which Geoweaver manages end-to-end deep learning for in-time crop mapping using Landsat data. We show how Geoweaver effectively removes the tedium of managing various scripts, code, libraries, Jupyter Notebooks, datasets, servers, and platforms, greatly reducing the time, cost, and effort researchers must spend on such AI-based workflows. The concepts demonstrated through Geoweaver serve as an important building block in the future of cyberinfrastructure for AI research.