Intelligent dynamic scheduling in clusters and computational grids

计算网格在加拿大的研究领域发挥着重要作用，因为它具有高度分布式的性质，并得到了几个组织和资助机构的支持，如GridCanada，C3，Canarie和CFI -例如，通过Sharcnet集群网络，为化学、工程和经济等高度多样化的应用领域提供服务。计算网格提供了连接和共享HPC资源的选项，以解决超出本地站点容量的大规模问题，或者平衡站点之间本地工作负载中的热点。对于大规模作业的调度，可以选择粗粒度工作流或同时资源分配，其中该建议解决了后者。因此，在不同的站点上找到匹配的调度时隙是困难的。此外，处理器必须处理异构性，现代多核CPU（如AMD Opteron）中提供的新多处理功能，以及新的I/O密集型应用程序（如数据挖掘）。这些趋势使作业调度方法是必要的，灵活和动态的，以提高机器的利用率，并找到同时的时隙，而不损害本地的工作量。本研究以适应性资源分配和分时作业执行的形式解决了这种动态方法。其中一些方法还需要中间件来使应用程序具有适应性。总而言之，在网格、作业和应用程序级别都需要创新的解决方案。本研究提出了自适应作业调度的解决方案，特别是分时作业调度的多核CPU，这些方法集成到网格调度，特殊的网格调度与经济模型，自适应中间件与高质量的数据分布的一类重要的通信关键的数字应用。这些解决方案将得到工具环境的实际支持，并在理论上得到高级成本建模的支持。此外，解决方案将被设计为可配置的应用程序和资源感知，并提取适用范围。

Computational grids play a major role in the Canadian research landscape due its highly distributed nature and are supported by several organizations and funding agencies like GridCanada, C3, Canarie, and CFI - e.g., via the Sharcnet network of clusters, serving highly diverse application areas like chemistry, engineering, and economics. Computational grids provide the option to join and share HPC resources for tackling large-scale problems that go beyond the local site's capacities or for balancing hot spots in local workloads among sites. For scheduling of large-scale jobs, either coarse-grain workflow or simultaneous resource allocation may be chosen, of which this proposal addresses the latter. Finding matching scheduling slots on different sites, then, is difficult. Furthermore, schedulers have to deal with heterogeneity, new multi-processing features available in modern multi-core CPUs like the AMD Opteron, and new I/O-intensive applications like datamining. These trens make job-scheduling approaches necessary that are flexible and dynamic to improve machine utilization and to find simultaneous time slots without harming local workload. This research addresses such dynamic approaches in the form of adaptive resource allocation and time-shared job execution. Some of these approaches also require middleware to make the applications adaptable. Altogether, innovative solutions are required at the grid, job, and application level. This research proposes solutions for adaptive job scheduling, special time-shared job scheduling on multi-core CPUs, an integration of these approaches into grid scheduling, special grid schedulers with economic models, and adaptive middleware with high-quality data distribution for an important class of communication-critical numeric applications. The solutions will be practically supported by a tool environment and theoretically by advanced cost modeling. Furthermore, the solutions will be designed configurable to be application- and resource-aware, and suitability ranges will be extracted.