Web technologies for environmental Big Data
Web technologies for environmental Big Data
复制标题
DOI：
10.1016/j.envsoft.2014.10.007
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
C. Vitolo;Yehia El-khatib;D. Reusser;C. Macleod;W. Buytaert
中科院分区：
文献类型：
--
作者：
C. Vitolo;Yehia El-khatib;D. Reusser;C. Macleod;W. Buytaert
计算科学和网络技术的最新发展为环境界提供了不断扩大的数据收集和分析资源，这对分析方法、工作流程以及与数据集交互的设计提出了前所未有的挑战。鉴于最近英国研究委员会资助的环境虚拟观测站试点项目，本文概述了与用于处理大型异构数据集的基于网络的技术相关的当前可用实施，并讨论了它们在环境数据处理、模拟和预测背景下的相关性。我们发现，使用 R、RPy2、PyWPS 和 PostgreSQL 的组合，试验中使用的简单数据集的处理相对简单。然而，使用 NoSQL 数据库和更通用的框架（例如基于 OGC 标准的实现）可以提供更广泛、更灵活的功能集，特别有助于处理更大容量和更异构的数据源。
Recent evolutions in computing science and web technology provide the environmental community with continuously expanding resources for data collection and analysis that pose unprecedented challenges to the design of analysis methods, workflows, and interaction with data sets. In the light of the recent UK Research Council funded Environmental Virtual Observatory pilot project, this paper gives an overview of currently available implementations related to web-based technologies for processing large and heterogeneous datasets and discuss their relevance within the context of environmental data processing, simulation and prediction. We found that, the processing of the simple datasets used in the pilot proved to be relatively straightforward using a combination of R, RPy2, PyWPS and PostgreSQL. However, the use of NoSQL databases and more versatile frameworks such as OGC standard based implementations may provide a wider and more flexible set of features that particularly facilitate working with larger volumes and more heterogeneous data sources.