EPypes: a framework for building event-driven data processing pipelines.
EPypes: a framework for building event-driven data processing pipelines.
复制标题
DOI：
10.7717/peerj-cs.176
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Falkman P
中科院分区：
文献类型：
--
作者：
Semeniuta O;Falkman P
许多数据处理系统自然地被建模为管道，其中数据流经计算过程网络。这种表示特别适合计算机视觉算法，在大多数情况下，计算机视觉算法具有复杂的逻辑和大量需要调整的参数。此外，在线视觉系统（例如工业自动化环境中的系统）必须与其他分布式节点进行通信。开发视觉系统时，通常从临时实验和原型设计到高度结构化的系统集成。这个连续体的早期阶段的特点是开发可行算法的挑战，而后者则涉及在网络环境中将视觉功能与其他组件组合在一起。在这两者之间，人们努力管理所开发系统的复杂性，并保存现有知识。为了应对这些挑战，本文提出了 EPypes，这是一种基于架构和 Python 的软件框架，用于以计算图的形式开发视觉算法及其与基于发布-订阅通信的分布式系统的集成。 EPypes 促进了算法原型设计的灵活性，并提供了一种结构化方法来管理算法逻辑并将开发的管道作为在线系统的一部分公开。
Many data processing systems are naturally modeled as pipelines, where data flows though a network of computational procedures. This representation is particularly suitable for computer vision algorithms, which in most cases possess complex logic and a big number of parameters to tune. In addition, online vision systems, such as those in the industrial automation context, have to communicate with other distributed nodes. When developing a vision system, one normally proceeds from ad hoc experimentation and prototyping to highly structured system integration. The early stages of this continuum are characterized with the challenges of developing a feasible algorithm, while the latter deal with composing the vision function with other components in a networked environment. In between, one strives to manage the complexity of the developed system, as well as to preserve existing knowledge. To tackle these challenges, this paper presents EPypes, an architecture and Python-based software framework for developing vision algorithms in a form of computational graphs and their integration with distributed systems based on publish-subscribe communication. EPypes facilitates flexibility of algorithm prototyping, as well as provides a structured approach to managing algorithm logic and exposing the developed pipelines as a part of online systems.