Reconfigurable Parallel Processing Plug＆Play Clustering

该项目的目标是在大规模集群中推动容错和可重构性的技术极限，使集群几乎可以自我维持，而重新配置是一个“即插即用”的问题。主要成果如下：1)构建了“即插即用”集群测试平台（20个节点采用DELL Inspiron，移动赛扬600mhz， 128mb内存，20gb硬盘，3COM即插即用PCMCIA 100Base-T网卡）。这可以作为中间件开发的灵活测试平台。它也非常紧凑（一个小机架）和低功耗（小于400瓦/20个节点）2)开发了长尾小鹦鹉容错，高性能集群MPI，允许用户根据他的应用特点从一组可用的算法中选择各种检查点算法。Parakeet是一个完全的用户级实现，可移植且高效，并将用户从代码中的检查点问题中解放出来。我们实现了多种检查点策略，以达到最佳效率，并与完全重启进行了详细的性能分析。3)自组织集群中间件，Lucie原型。即插即用集群作为一种基础技术，需要节点的热插拔、节点内软件组织的重新配置和动态分区管理。Lucie构建在现有的Linux工具之上，以自动化的方式实现完整的集群配置功能。Lucie允许以非常快速的方式对集群中的每个节点进行全自动（重新）安装和配置。4)可扩展、安全和自组织的集群通信原型。我们已经确定，集群节点内可扩展、可靠、安全和自组织的通信是可靠的即插即用集群的必要基础。我们已经在Gfarm（用于Petascale Datagrid处理的集群中间件）作业管理器中对一些想法进行了原型化：在Gfarm中，自组织进程环结构管理所有节点，并且作业可以以安全的方式快速并行启动。少

The objective of the project is to push the technological envelop of fault tolerance and reconfigurability in large-scale clustering such that the clusters become almost self-sustaining, and reconfiguring is a matter of "Plug&Play". Some of the salient results are as follows :1) Construction of the "Plug&Play" clustering testbed (20 nodes of DELL Inspiron , Mobile Celeron 600 MHz, 128 MB Memory, 20 GB HDD, 3COM Plug＆Play PCMCIA 100Base-T Network Card). This served as a flexible testbed for middleware development. It was also very compact (a small rack) and low power (less than 400 watts/20 nodes)2) Development of the Parakeet Fault Tolerant, High-Performance Cluster MPI which allows various checkpointing algorithms to be selected from a set of available algorithm by the user according to his application characteristics. Parakeet is an entirely user-level implementation, is portable and efficient, and frees the users from checkpointing concerns within his code. We have implemented vario … More us checkpointing strategies to achieve the best efficiency, and conducted detailed performance analysis comparing with full restart.3) Self-organizing cluster middleware, the Lucie prototype. As a basic technology, plug＆play clustering requires hot swapping of nodes, reconfiguration of software organization within a node, and dynamic partition management. Lucie builds on existing Linux tools to implement full cluster configuration capabilities in an automated fashion. Lucie allows fully automated (re)installation and configuration of every node in a cluster in a very rapid fashion.4) Prototyping scalable, secure and self-organizing cluster communication. We have identified that scalable, reliable, secure, and self-organizing communication within the cluster node is the essential foundation for reliable, plug＆play clustering. We have prototyped some of the ideas in the Gfarm (cluster middleware for Petascale Datagrid processing) job manager : there, the self-organizing process ring structure governs all the nodes, and jobs can be started up rapidly in parallel, in a safe secure manner. Less