A framework for reliable and efficient data placement in distributed computing systems
A framework for reliable and efficient data placement in distributed computing systems
复制标题
DOI：
10.1016/j.jpdc.2005.04.019
复制
发表时间：
2005-10-01
影响因子：
3.8
通讯作者：
Livny, M
中科院分区：
文献类型：
--
作者：
Kosar, T;Livny, M
数据放置是当今分布式应用程序的重要组成部分，因为将数据移动到应用程序附近有许多好处。科学和商业应用日益增长的数据需求，以及对这些数据的协作访问使其更加重要。在目前的方法中，数据放置被认为是计算的一个副作用。我们的目标是使数据放置成为分布式计算系统中的头等公民，就像计算作业一样。它们将被排队、调度、监视、管理，甚至检查点。由于数据放置作业与计算作业具有不同的特征，因此不能以与计算作业完全相同的方式对待它们。为此，我们提出了一个框架，它可以被视为分布式计算系统的“数据放置子系统”，类似于操作系统中的I/O子系统。这个框架包括一个专门用于数据放置的调度器、一个了解数据放置作业的高级规划器、一个资源代理/策略执行器和一些优化工具。我们的系统能够进行可靠高效的数据放置，能够在不需要任何人为干预的情况下从各种故障中恢复，并且能够在执行时动态适应环境。(c) 2005爱思唯尔公司
Data placement is an essential part of today's distributed applications since moving the data close to the application has many benefits. The increasing data requirements of both scientific and commercial applications, and collaborative access to these data make it even more important. In the current approach, data placement is regarded as a side affect of computation. Our goal is to make data placement a first class citizen in distributed computing systems just like the computational jobs. They will be queued, scheduled, monitored, managed, and even checkpointed. Since data placement jobs have different characteristics than computational jobs, they cannot be treated in the exact same way as computational jobs. For this purpose, we are proposing a framework which can be considered as a "data placement subsystem" for distributed computing systems, similar to the I/O subsystem in operating systems. This framework includes a specialized scheduler for data placement, a high level planner aware of data placement jobs, a resource broker/policy enforcer and some optimization tools. Our system can perform reliable and efficient data placement, it can recover from all kinds of failures without any human intervention, and it can dynamically adapt to the environment at the execution time. (c) 2005 Elsevier Inc.