M-TOP: multi-target operator placement of query graphs for data streams
M-TOP: multi-target operator placement of query graphs for data streams
复制标题
M-TOP：数据流查询图的多目标运算符放置
DOI：
--
复制
发表时间：
2011
期刊：
影响因子：
--
通讯作者：
B. Mitschang
中科院分区：
文献类型：
--
作者：
N. Cipriani;O. Schiller;B. Mitschang
如今，许多应用程序处理基于流的数据，如金融市场分析，网络入侵检测或可视化应用程序。为了以独立于应用程序的方式处理基于流的数据，出现了分布式流处理系统。它们通常将查询转换为运算符图，将运算符放置到流处理节点，并执行它们来处理流数据。在这样的系统中，操作符的放置是至关重要的，因为它深刻地影响查询执行。通常，不同的基于流的应用需要根据它们的特定目标（例如，带宽不小于500 MBit/s并且成本不超过1个成本单位）来专门放置查询图。这一事实限制了操作员的位置。现有的方法不考虑应用程序特定的目标，因此不反映应用程序特定的布局决策。由于目标之间可能相互冲突，运营商的布局受到微妙的权衡，如带宽最大化比成本降低更重要。因此，面临的挑战是找到一个解决方案，考虑到特定于应用程序的目标和他们的权衡。
我们提出了M-TOP，一个QoS感知的数据流系统的多目标运营商布局框架。特别是，我们提出了一个运营商的位置策略，考虑到特定于应用程序的目标，其各自的权衡规格，瓶颈条件和排名计划，以计算一个合适的位置。我们将M-TOP集成到我们的分布式数据流处理中间件NexusDS中，并提供了一个实验评估来显示M-TOP的有效性。
Nowadays, many applications processes stream-based data, such as financial market analysis, network intrusion detection, or visualization applications. To process stream-based data in an application-independent manner, distributed stream processing systems emerged. They typically translate a query to an operator graph, place the operators to stream processing nodes, and execute them to process the streamed data. The operator placement is crucial in such systems, as it deeply influences query execution. Often, different stream-based applications require dedicated placement of query graphs according to their specific objectives, e.g. bandwidth not less than 500 MBit/s and costs not more that 1 cost unit. This fact constraints operator placement. Existing approaches do not take into account application-specific objectives, thus not reflecting application-specific placement decisions. As objectives might conflict among each other, operator placement is subject to delicate trade-offs, such as bandwidth maximization is more important than cost reduction. Thus, the challenge is to find a solution which considers the application-specific objectives and their trade-offs.
We present M-TOP, an QoS-aware multi-target operator placement framework for data stream systems. Particularly, we propose an operator placement strategy considering application-specific targets consisting of objectives, their respective trade-offs specifications, bottleneck conditions, and ranking schemes to compute a suitable placement. We integrated M-TOP into NexusDS, our distributed data stream processing middleware, and provide an experimental evaluation to show the effectiveness of M-TOP.