A Software Chain Approach to Big Data Stream Processing and Analytics
A Software Chain Approach to Big Data Stream Processing and Analytics
复制标题
大数据流处理和分析的软件链方法
DOI：
10.1109/cisis.2015.24
复制
发表时间：
2015
期刊：
影响因子：
--
通讯作者：
L. Barolli
中科院分区：
文献类型：
--
作者：
F. Xhafa;V. Naranjo;S. Caballé;L. Barolli
大数据流处理是当今最重要的计算趋势之一。对大数据流处理的兴趣越来越大，这是因为许多基于互联网的应用程序需要生成巨大的数据流，这些数据流的处理可以用于提取有用的分析并为决策系统提供信息。例如，用于供应链的基于IoT的监控系统可以为业务交付性能提供真实的时间数据分析。处理大数据流的挑战在于应对无限数据流的实时处理，也就是说，计算系统应该能够以高吞吐量计算以适应输入中的高数据流速率生成。显然，数据流速率越高，吞吐量就应该越高，以实现处理结果的一致性（例如，保持数据流中事件的顺序）。在本文中，我们将展示如何映射的数据流处理阶段（从数据生成到最终结果）的软件链架构，其中包括五个主要组成部分：传感器，提取器，解析器，格式化器和输出推杆。我们使用Yahoo！S4用于处理来自Flight Radar 24全球飞行监控系统的大数据流。
Big Data Stream processing is among the most important computing trends nowadays. The growing interest on Big Data Stream processing comes from the need of many Internet-based applications that generate huge data streams, whose processing can serve to extract useful analytics and inform for decision making systems. For instance, an IoT-based monitoring systems for a supply-chain, can provide real time data analytics for the business delivery performance. The challenges of processing Big Data Streams reside on coping with real-time processing of an unbounded stream of data, that is, the computing system should be able to compute at high throughput to accommodate the high data stream rate generation in input. Clearly, the higher the data stream rate, the higher should be the throughput to achieve consistency of the processing results (e.g. Preserving the order of events in the data stream). In this paper we show how to map the data stream processing phases (from data generation to final results) to a software chain architecture, which comprises five main components: sensor, extractor, parser, formatter and out putter. We exemplify the approach using the Yahoo!S4 for processing the Big Data Stream from Flight Radar24 global flight monitoring system.