Parallel complex event processing to meet probabilistic latency bounds II

满足概率延迟界限的并行复杂事件处理 II

• 批准号: 268797282
• 负责人: Professorin Dr. Sukanya Bhowmik
• 金额: --
• 依托单位: Institut für Parallele und Verteilte Systeme (IPVS)
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2015
• 资助国家: 德国
• 起止时间: 2014-12-31 至 2022-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/268797282?language=en
• 关键词: Parallel; complex; event; processing; meet

Meeting latency bounds in complex event processing (CEP) in a consistent and cost-efficient manner is highly challenging. Operators have to be capable of processing events at very high rate and also under varying workload. This requires methods to parallelize multiple dependent operators and dynamically adapt the configuration of these operators, e.g., by changing the parallelization degree and resources used in each operators' configuration. As a result, in the first phase of this project (Precept I) we developed methods for fine-grained parallelization of CEP operators that can probabilistically meet a latency bound under heavily fluctuating and dynamic workloads. In Precept I, we approached the problem under the assumption that there are unlimited resources available. However, there are situations where the monetary budget and/or the available resources for running a CEP system are limited and additional mechanisms must be employed to meet the lateny bounds of the system. In order to address the aforementioned problem, in this proposal (Precept II), we aim to shed load at the operators so that a given latency bound for a CEP system can be met despite resource constraints. While shedding load may have an adverse impact on the quality of complex event processing, we aim to develop load shedding strategies for operators that maximize the quality of result perceived by the application and also meet the required latency bounds. Our load shedding strategies should support a wide spectrum of CEP operators. Furthermore, we aim to combine the optimization scheme without load shedding (Precept I) and the scheme with load shedding (Precept II) such that, for any input condition, the system maximizes quality while maintaining the given latency bound, and minimizes cost while the resource limit is not reached. The proposed methods and concepts will be evaluated using real-world workloads of camera networks and traffic monitoring.

以一致且经济高效的方式满足复杂事件处理(CEP)的延迟限制是非常具有挑战性的。操作员必须能够以非常高的速度处理事件，并且能够在不同的工作负载下处理事件。这需要对多个从属运算符进行并行化并动态调整这些运算符的配置的方法，例如，通过改变每个运算符的配置中使用的并行化程度和资源。因此，在这个项目的第一阶段(方案I)，我们开发了CEP操作符的细粒度并行化方法，在剧烈波动和动态的工作负载下，这些方法可以概率地满足延迟界限。在方案一中，我们在假设有无限的可用资源的情况下处理了这个问题。然而，在某些情况下，货币预算和/或用于运行CEP系统的可用资源是有限的，必须采用额外的机制来满足该系统的滞后界限。为了解决上述问题，在本提案(方案II)中，我们的目标是减轻运营商的负担，以便在不考虑资源限制的情况下满足CEP系统的给定延迟限制。虽然减负可能会对复杂事件处理的质量产生不利影响，但我们的目标是为操作员开发减负策略，以最大化应用程序感知的结果质量，并满足所需的延迟界限。我们的减负策略应该支持广泛的CEP运营商。此外，我们的目标是将无卸载优化方案(方案I)和有卸载优化方案(方案II)相结合，使得对于任何输入条件，系统在保持给定延迟界限的同时最大化质量，在不达到资源限制的情况下最小化成本。建议的方法和概念将使用摄像机网络和交通监控的真实工作负载进行评估。