Taskscheduling for distributed programs which include streaming processing

我们研究的目的是执行流应用，例如，分析从网络源提供的流数据和音频和视频流的处理，通过调度它们来有效地利用计算资源，如处理器和存储，这些计算资源可能包括在并行和分布式环境中，如网格或PC集群。包括流处理在内的分布式应用不能有效地执行，因为几乎所有传统的调度方法都不能直接应用于它们。我们提出了一种新的任务调度方法来调度这些流媒体应用。我们还使用自行开发的分布式网络监控系统对该方法进行了评估。该分布式网络监控系统是一个实际的流媒体应用。它监视和缓存IP数据报以检查内容的安全性。这种应用比安全设备更便宜，因为它可以改进安全检查是通过利用包括空闲PC的PC集群来实现的。此外，与安全设备相比，它非常灵活，因为它可以很容易地针对新的安全问题添加新的功能。该方法可以很好地同时调度流媒体应用和非流媒体应用。实验结果表明，该方法减少了实际流应用的总执行时间。即使网络带宽是动态变化的，执行时间的分散也被抑制。当该方法应用于更复杂的任务调度方法时，这一特性是非常重要的。

The purpose of our study is to execute streaming applications, for example, analyzing stream data provided from network sources and processing of audio and video stream, effectively by scheduling them to utilize computing resources like processors and storages which are potentially comprised in parallel and distributed environments like Grids or PC clusters. Distributed applications including streaming processing cannot be executed efficiently, because almost all conventional scheduling methods are not able to apply to them directly. We proposed a new task scheduling method which can schedule these streaming applications. We also use a distributed network monitoring system which developed by us for an evaluation of the proposed method. This distributed network monitoring system is one of practical streaming application. It monitors and caches IP datagram to check the contents for security. This application is cheaper than the security appliances, because it can improve that security checking is realized by utilizing PC clusters which includes idle PCs. Moreover it is very flexible compared with the security appliances, because it can append to a new function against a new security problem easily. The proposed method can schedule both streaming application and non-streaming applications well at a same time. The experimental results showed the proposed method reduced the total execution time of the practical stream application. The dispersion of the execution time was also suppressed even if the network bandwidth was dynamically changing. This characteristic is very important when this method is applied to more complicated task scheduling methods.