Parallel complex event processing to meet probabilistic latency bounds II

以一致且经济高效的方式满足复杂事件处理（CEP）中的延迟界限是非常具有挑战性的。操作人员必须能够以非常高的速率处理事件，并在不同的工作量下处理事件。这需要并行化多个相关操作符并动态调整这些操作符的配置的方法，例如，通过改变每个操作符配置中使用的并行化程度和资源。因此，在该项目的第一阶段（戒律1）中，我们开发了用于细粒度并行化CEP操作符的方法，这些方法可能会在剧烈波动和动态工作负载下满足延迟限制。在戒律1中，我们在存在无限可用资源的假设下处理问题。然而，在某些情况下，用于运行CEP系统的货币预算和/或可用资源是有限的，必须采用额外的机制来满足系统的延迟界限。为了解决上述问题，在本建议（守则II）中，我们的目标是减轻营办商的负担，以便在资源有限的情况下，仍能满足CEP系统的既定延迟界限。虽然减载可能对复杂事件处理的质量产生不利影响，但我们的目标是为运营商开发减载策略，以最大限度地提高应用程序感知的结果质量，并满足所需的延迟界限。我们的减载策略应支援各种CEP营运商。进一步，我们的目标是将无减载优化方案（规则I）和有减载优化方案（规则II）结合起来，使得对于任何输入条件，系统在保持给定延迟界限的情况下实现质量最大化，在不达到资源限制的情况下实现成本最小化。所提出的方法和概念将使用摄像机网络和交通监控的实际工作负载进行评估。

Meeting latency bounds in complex event processing (CEP) in a consistent and cost-efficient manner is highly challenging. Operators have to be capable of processing events at very high rate and also under varying workload. This requires methods to parallelize multiple dependent operators and dynamically adapt the configuration of these operators, e.g., by changing the parallelization degree and resources used in each operators' configuration. As a result, in the first phase of this project (Precept I) we developed methods for fine-grained parallelization of CEP operators that can probabilistically meet a latency bound under heavily fluctuating and dynamic workloads. In Precept I, we approached the problem under the assumption that there are unlimited resources available. However, there are situations where the monetary budget and/or the available resources for running a CEP system are limited and additional mechanisms must be employed to meet the lateny bounds of the system. In order to address the aforementioned problem, in this proposal (Precept II), we aim to shed load at the operators so that a given latency bound for a CEP system can be met despite resource constraints. While shedding load may have an adverse impact on the quality of complex event processing, we aim to develop load shedding strategies for operators that maximize the quality of result perceived by the application and also meet the required latency bounds. Our load shedding strategies should support a wide spectrum of CEP operators. Furthermore, we aim to combine the optimization scheme without load shedding (Precept I) and the scheme with load shedding (Precept II) such that, for any input condition, the system maximizes quality while maintaining the given latency bound, and minimizes cost while the resource limit is not reached. The proposed methods and concepts will be evaluated using real-world workloads of camera networks and traffic monitoring.