PAIRS: A scalable geo-spatial data analytics platform
PAIRS: A scalable geo-spatial data analytics platform
复制标题
PAIRS：可扩展的地理空间数据分析平台
DOI：
--
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
H. Hamann
中科院分区：
文献类型：
--
作者：
L. Klein;F. Marianno;C. Albrecht;Marcus Freitag;Siyuan Lu;Nigel Hinds;X. Shao;Sergio Bermudez Rodriguez;H. Hamann
地理空间数据量超过数百PB，并且主要由移动的设备和高分辨率成像系统生成的图像/视频/数据驱动呈指数级增长。在历史档案和/或真实的时间数据集上的快速数据发现目前受到具有不同投影和空间分辨率的各种数据格式的限制，在可以执行分析之前需要大量的数据处理。提出了一种称为物理分析集成存储库和服务（PAIRS）的新平台，该平台通过在空间和时间上自动更新，加入和重新定位数据层来实现快速数据发现。PAIRS建立在开源大数据软件之上，管理自动数据下载、数据管理和可扩展存储，同时也是一个计算平台，用于在管理的数据集上运行物理和统计模型。通过在数据上传到平台之前解决数据策展，可以真实的实时执行多层查询和过滤。此外，PAIRS为开发定制分析提供了基础。为此，我们提出了两个运行模型的例子：（1）高分辨率蒸散和农业植被监测，以及（2）由机器学习驱动的超本地天气预报，用于可再生能源预测。
Geospatial data volume exceeds hundreds of Petabytes and is increasing exponentially mainly driven by images/videos/data generated by mobile devices and high resolution imaging systems. Fast data discovery on historical archives and/or real time datasets is currently limited by various data formats that have different projections and spatial resolution, requiring extensive data processing before analytics can be carried out. A new platform called Physical Analytics Integrated Repository and Services (PAIRS) is presented that enables rapid data discovery by automatically updating, joining, and homogenizing data layers in space and time. Built on top of open source big data software, PAIRS manages automatic data download, data curation, and scalable storage while being simultaneously a computational platform for running physical and statistical models on the curated datasets. By addressing data curation before data being uploaded to the platform, multi-layer queries and filtering can be performed in real time. In addition, PAIRS offers a foundation for developing custom analytics. Towards that end we present two examples with models which are running operationally: (1) high resolution evapo-transpiration and vegetation monitoring for agriculture and (2) hyperlocal weather forecasting driven by machine learning for renewable energy forecasting.