CSR: Small:Data Staging and Parallel Applications in Robust Desktop Grids

该项目是对算法和机制的研究，这些算法和机制将允许在桌面网格中进行复杂的输入和输出数据。数据分段允许系统将数据半永久地存储在底层的对等结构中，并运行多节点作业（应用程序），无论它们是紧密耦合的并行应用程序、任意工作流，还是介于两者之间的任何东西。通过在三个不同的领域扩展当前的桌面网格基础设施，我们正在构建对这些应用程序类型的支持。我们正在开发集群识别技术，可以通过被动和主动网络测量来定义任意大小的虚拟集群。我们正在将虚拟集群描述合并到底层点对点基础设施中，以允许调度算法将多节点作业映射到集群。最后，我们正在将数据放置整合到底层基础设施中；数据是根据使用和进程绑定放置的。这项工作将影响几个研究领域，包括分布式和分散调度、应用程序描述、网络表征和存储网络。在所有这些领域，我们的工作将探索地方自治与全球总体目标之间的紧张关系。这些算法和技术将在广泛的新兴分布式和协作应用中具有广泛的适用性。然而，这里描述的工作也将明确和立即影响我们在天文学和其他领域的合作者所进行的研究质量。运行并行应用程序的能力，以及那些具有更复杂相互关系的应用程序的能力，将使全新的科学应用程序能够在特设的类似网格的系统上运行。

This project is an investigation into algorithms and mechanisms that will allow sophisticated staging of input and output data in desktop grids. Data staging allows the system to store data semi-permanently in the underlying peer-to-peer structure, and to run multi-node jobs (applications) whether they be tightly-coupled parallel applications, arbitrary work-flows, or anything between.We are building support for these application types by extending our current desktop grid infrastructure in three distinct areas. We are developing cluster identification techniques that can define arbitrarily-sized virtual clusters through both passive and active network measurement. We are incorporating virtual cluster descriptions into the underlying peer-to-peer infrastructure to allow the scheduling algorithms to map multi-node jobs to the clusters. Finally, we are incorporating data placement into the underlying infrastructure; data is placed according to use and process binding.This work will impact several research areas, including that of distributed and decentralized scheduling, application description, network characterization, and storage networks. In all of these areas our work will explore the tension between local autonomy and global, aggregate objectives.The algorithms and techniques will have broad applicability across a wide range of emerging distributed and collaborative applications. However, the work described here will also explicitly and immediately impact the quality of research conducted by our collaborators in astronomy and elsewhere. The ability to run parallel applications, and those with more complicated inter-relationships, will enable whole new classes of scientific applications to be run on top of ad-hoc grid-like systems.