Benchflow - A Benchmark for Workflow Management Systems

Benchflow - 工作流程管理系统的基准

• 批准号: 229974593
• 负责人: Professor Dr. Frank Leymann
• 金额: --
• 依托单位: Faculty of Informatics
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2013
• 资助国家: 德国
• 起止时间: 2012-12-31 至 2016-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/229974593?language=en
• 关键词: Benchflow; Benchmark; Workflow; Management; Systems

The goal of the BenchFlow project is to design the first benchmark for assessing and comparing the performance of workflow management systems (aka business process execution engines). Given the large number of heterogeneous systems and languages that have been proposed for modeling and executing workflows, to ensure the feasibility of this project, we will initially focus our efforts on a benchmark for standard WS-BPEL compliant Web service composition engines. Due to the inherent complexity of workflow engine architectures and the very large number of parameters affecting their performance, benchmarking workflow systems poses a number of scientific research challenges that require investigating a novel set of performance evaluation techniques. These will complement existing ones used to benchmark database management systems (e.g., TPC) or programming language compilers (e.g., SPEC), which are already well understood and have contributed to drive a very large performance improvement within the database, compiler and processor architecture industry. Workflow systems have become the platform to build composite service-oriented applications, whose performance depends on two factors: the performance of the workflow system itself and the performance of the composed services (which could lie outside of the control of the workflow). We plan to use a model-driven, self-/recursive testing approach to eliminate the impact of the external services by having them implemented as processes. The processes selected for the benchmark will be synthesized out of real-world WS-BPEL processes. Given the very large number of metrics that can be used to observe the performance of a workflow system, we aim at distilling a reduced set of performance indicators to compare different engines as well as different configurations of a given engine.

BenchFlow项目的目标是设计第一个基准，用于评估和比较工作流管理系统(也称为业务流程执行引擎)的性能。鉴于已经提出了大量用于建模和执行工作流的异类系统和语言，为了确保该项目的可行性，我们首先将重点放在标准WS-BPEL兼容的Web服务组合引擎的基准测试上。由于工作流引擎体系结构的内在复杂性和影响其性能的大量参数，对工作流系统进行基准测试提出了许多科学研究挑战，需要研究一套新颖的性能评估技术。这些将补充现有的用于对数据库管理系统(例如TPC)或编程语言编译器(例如SPEC)进行基准测试的工具，这些工具已经被很好地理解，并有助于推动数据库、编译器和处理器体系结构行业内非常大的性能改进。工作流系统已经成为构建面向服务的组合应用的平台，其性能取决于两个因素：工作流系统本身的性能和组合服务的性能(这可能超出工作流的控制范围)。我们计划使用模型驱动的自/递归测试方法，通过将外部服务实现为流程来消除它们的影响。为基准选择的流程将从真实的WS-BPEL流程合成。考虑到可以用来观察工作流系统性能的大量指标，我们的目标是提取一组精简的性能指标来比较不同的引擎以及给定引擎的不同配置。