Real-Time Big Data Analytical Architecture for Remote Sensing Application
Real-Time Big Data Analytical Architecture for Remote Sensing Application
复制标题
DOI：
10.1109/jstars.2015.2424683
复制
发表时间：
2015-10-01
影响因子：
5.5
通讯作者：
Ji, Wen
中科院分区：
文献类型：
--
作者：
Rathore, Muhammad Mazhar Ullah;Paul, Anand;Ji, Wen
遥感数字世界的资产每天都会产生大量的实时数据（主要指“大数据”），其中的洞察信息如果得到有效的收集和聚合，将具有潜在的意义。在当今时代，实时遥感大数据比最初看起来要多得多，以有效的方式提取有用信息会导致系统面临重大计算挑战，例如分析、聚合和存储远程收集的数据。考虑到上述因素，需要设计一种既支持实时数据处理又支持离线数据处理的系统架构。因此，在本文中，我们提出了用于遥感卫星应用的实时大数据分析架构。所提出的架构包括三个主要单元，例如1）遥感大数据采集单元（RSDU）； 2）数据处理单元（DPU）； 3）数据分析决策单元（DADU）。首先，RSDU 从卫星获取数据并将该数据发送到基站，并在基站进行初始处理。其次，DPU 通过提供过滤、负载平衡和并行处理，在实时大数据高效处理的架构中发挥着至关重要的作用。第三，DADU是该架构的上层单元，负责编译、存储结果，并根据从DPU接收到的结果生成决策。所提出的架构具有仅有用数据的划分、负载平衡和并行处理的能力。因此，它可以利用地球观测系统有效地分析实时遥感大数据。此外，所提出的架构能够存储传入的原始数据，以便在需要时对大量存储的转储执行离线分析。最后，利用Hadoop对陆地和海域遥感地球观测大数据进行了详细分析。此外，还为 RSDU、DPU 和 DADU 的各个级别提出了各种算法来检测陆地和海域，以详细说明架构的工作原理。
The assets of remote senses digital world daily generate massive volume of real-time data (mainly referred to the term "Big Data"), where insight information has a potential significance if collected and aggregated effectively. In today's era, there is a great deal added to real-time remote sensing Big Data than it seems at first, and extracting the useful information in an efficient manner leads a system toward a major computational challenges, such as to analyze, aggregate, and store, where data are remotely collected. Keeping in view the above mentioned factors, there is a need for designing a system architecture that welcomes both real-time, as well as offline data processing. Therefore, in this paper, we propose real-time Big Data analytical architecture for remote sensing satellite application. The proposed architecture comprises three main units, such as 1) remote sensing Big Data acquisition unit (RSDU); 2) data processing unit (DPU); and 3) data analysis decision unit (DADU). First, RSDU acquires data from the satellite and sends this data to the Base Station, where initial processing takes place. Second, DPU plays a vital role in architecture for efficient processing of real-time Big Data by providing filtration, load balancing, and parallel processing. Third, DADU is the upper layer unit of the proposed architecture, which is responsible for compilation, storage of the results, and generation of decision based on the results received from DPU. The proposed architecture has the capability of dividing, load balancing, and parallel processing of only useful data. Thus, it results in efficiently analyzing real-time remote sensing Big Data using earth observatory system. Furthermore, the proposed architecture has the capability of storing incoming raw data to perform offline analysis on largely stored dumps, when required. Finally, a detailed analysis of remotely sensed earth observatory Big Data for land and sea area are provided using Hadoop. In addition, various algorithms are proposed for each level of RSDU, DPU, and DADU to detect land as well as sea area to elaborate the working of an architecture.