A Containerized Big Data Streaming Architecture for Edge Cloud Computing on Clustered Single-board Devices
A Containerized Big Data Streaming Architecture for Edge Cloud Computing on Clustered Single-board Devices
复制标题
集群单板设备边缘云计算的容器化大数据流架构
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
C. Pahl
中科院分区：
文献类型：
--
作者：
Remo Scolati;Ilenia Fronza;Nabil El Ioini;Areeg Samir;C. Pahl
物联网（IoT）设备产生的数据量不断增加，为支持云基础设施带来了挑战，云基础设施通常用于处理和存储数据。这项工作的重点是一种基于边缘云计算模型的替代方法，即在将数据传输到后台云基础设施之前对其进行处理和过滤。我们描述了一种低功耗和低成本的单板计算机集群（SBC）的实现，应用大数据领域的模型和技术，目的是减少必须转移到其他地方的数据量。为了实现该系统，构建了一个树莓派集群，依靠Docker容器化和部署Apache Hadoop和Apache Spark集群，然后在其上执行测试应用程序。基于Prometheus的监视堆栈（Prometheus是云原生行业中流行的监视和警报工具）用于收集系统指标并分析设置的性能。我们对系统的复杂性进行了评估，结果表明，通过集装箱化可以提高系统的容错性和易于维护，这使得所提出的解决方案适用于工业环境。此外，为了讨论所提议的体系结构的能力和局限性，本文还分析了整体性能，考虑了所提议的解决方案的资源使用以及设备所施加的约束。
The constant increase of the amount of data generated by Internet of Things (IoT) devices creates challenges for the supporting cloud infrastructure, which is often used to process and store the data. This work focuses on an alternative approach, based on the edge cloud computing model, i.e., processing and filtering data before transferring it to a backing cloud infrastructure. We describe the implementation of a low-power and low-cost cluster of single board computers (SBC) for this context, applying models and technologies from the Big Data domain with the aim of reducing the amount of data which has to be transferred elsewhere. To implement the system, a cluster of Raspberry Pis was built, relying on Docker to containerize and deploy an Apache Hadoop and Apache Spark cluster, on which a test application is then executed. A monitoring stack based on Prometheus, a popular monitoring and alerting tool in the cloud-native industry, is used to gather system metrics and analyze the performance of the setup. We evaluate the complexity of the system, showing that by means of containerization increased fault tolerance and ease of maintenance can be achieved, which makes the proposed solution suitable for an industrial environment. Furthermore, an analysis of the overall performance, which takes into account the resource usage of the proposed solution with regards to the constraints imposed by the devices, is presented in order to discuss the capabilities and limitations of proposed architecture.