A Flexible IoT Stream Processing Architecture Based on Microservices
A Flexible IoT Stream Processing Architecture Based on Microservices
复制标题
DOI：
10.3390/info11120565
复制
发表时间：
2020-12-01
期刊：
影响因子：
3.1
通讯作者：
Rulli, Matteo
中科院分区：
文献类型：
--
作者：
Bixio, Luca;Delzanno, Giorgio;Rulli, Matteo
物联网(IoT)为数据分析创造了新的、具有挑战性的机遇。物联网代表着海量和异质数据的无限来源，其实时处理是一个日益重要的问题。物联网应用程序通常由多个技术层组成，这些层将“事物”连接到远程云核心。这些层通常分为两个宏观级别：边缘级别(由网络边界上的设备组成，靠近产生数据的设备)和核心层(由应用程序的远程云组件组成)。本工作的目的是提出一种适用于物联网平台的自适应微服务架构，该架构提供能够在边缘级和云级无缝连接的实时流处理功能。更详细地，我们引入了MU服务的概念，即可以在边缘和核心级上无关紧要地分配的流处理单元，以及提供以非常灵活的方式处理实时流处理的所有必要服务(即代理、适配器和数据处理MU服务)的参考体系结构。此外，为了从底层流处理引擎和物联网(边缘/云)中抽象出来，我们提出了：(1)服务定义语言，包括基于JSON对象的配置语言(互操作性)；(2)基于规则的查询语言，具有基本的过滤操作，可以编译到大多数现有的流处理引擎(可移植性)；(3)组合器语言，用于构建过滤器定义的管道(组合性)。尽管我们的提案旨在扩展由FlairBit开发的专有物联网平台Senseioty平台，但它可以适应基于类似技术的每个平台。作为概念验证，我们提供了一个基于Java OSGi框架的初步原型的详细信息。
The Internet of Things (IoT) has created new and challenging opportunities for data analytics. The IoT represents an infinitive source of massive and heterogeneous data, whose real-time processing is an increasingly important issue. IoT applications usually consist of multiple technological layers connecting 'things' to a remote cloud core. These layers are generally grouped into two macro levels: the edge level (consisting of the devices at the boundary of the network near the devices that produce the data) and the core level (consisting of the remote cloud components of the application). The aim of this work is to propose an adaptive microservices architecture for IoT platforms which provides real-time stream processing functionalities that can seamlessly both at the edge-level and cloud-level. More in detail, we introduce the notion of mu-service, a stream processing unit that can be indifferently allocated on the edge and core level, and a Reference Architecture that provides all necessary services (namely Proxy, Adapter and Data Processing mu-services) for dealing with real-time stream processing in a very flexible way. Furthermore, in order to abstract away from the underlying stream processing engine and IoT layers (edge/cloud), we propose: (1) a service definition language consisting of a configuration language based on JSON objects (interoperability), (2) a rule-based query language with basic filter operations that can be compiled to most of the existing stream processing engines (portability), and (3) a combinator language to build pipelines of filter definitions (compositionality). Although our proposal has been designed to extend the Senseioty platform, a proprietary IoT platform developed by FlairBit, it could be adapted to every platform based on similar technologies. As a proof of concept, we provide details of a preliminary prototype based on the Java OSGi framework.