Reconfigurable Parallel Processing Plug＆Play Clustering

该项目的目标是推动大规模集群中容错和可重构性的技术发展，使集群几乎能够自我维持，而重构是一个“即插即用”的问题。1）构建了“Plug&Play”集群实验床（20个节点的DELL网卡、600 MHz的移动的网卡、128 MB内存、20 GB硬盘、3COM Plug&Play PCMCIA 100 Base-T网卡）。这是中间件开发的一个灵活的测试平台。它也非常紧凑（一个小机架）和低功耗（小于400瓦/20个节点）2）开发的鹦鹉容错，高性能集群MPI，允许各种检查点算法从一组可用的算法由用户根据他的应用程序的特点选择。Parakeet是一个完全用户级的实现，可移植且高效，并将用户从代码中的检查点问题中解放出来。我们已经实现了vario 关于我们 采用检查点策略达到最佳效率，并与完全重启进行了详细的性能分析比较。3）自组织集群中间件Lucie原型。作为一种基本技术，即插即用集群需要节点的热插拔、节点内软件组织的重新配置以及动态分区管理。Lucie以现有的Linux工具为基础，以自动化的方式实现完整的集群配置功能。Lucie允许以非常快速的方式完全自动化（重新）安装和配置集群中的每个节点。4）原型化可扩展，安全和自组织的集群通信。我们已经确定，集群节点内可伸缩、可靠、安全和自组织的通信是可靠的即插即用集群的重要基础。我们已经在Gfarm（Petascale Datagrid处理的集群中间件）作业管理器中原型化了一些想法：在那里，自组织进程环结构管理所有节点，作业可以以安全的方式快速并行启动。少

The objective of the project is to push the technological envelop of fault tolerance and reconfigurability in large-scale clustering such that the clusters become almost self-sustaining, and reconfiguring is a matter of "Plug&Play". Some of the salient results are as follows :1) Construction of the "Plug&Play" clustering testbed (20 nodes of DELL Inspiron , Mobile Celeron 600 MHz, 128 MB Memory, 20 GB HDD, 3COM Plug＆Play PCMCIA 100Base-T Network Card). This served as a flexible testbed for middleware development. It was also very compact (a small rack) and low power (less than 400 watts/20 nodes)2) Development of the Parakeet Fault Tolerant, High-Performance Cluster MPI which allows various checkpointing algorithms to be selected from a set of available algorithm by the user according to his application characteristics. Parakeet is an entirely user-level implementation, is portable and efficient, and frees the users from checkpointing concerns within his code. We have implemented vario … More us checkpointing strategies to achieve the best efficiency, and conducted detailed performance analysis comparing with full restart.3) Self-organizing cluster middleware, the Lucie prototype. As a basic technology, plug＆play clustering requires hot swapping of nodes, reconfiguration of software organization within a node, and dynamic partition management. Lucie builds on existing Linux tools to implement full cluster configuration capabilities in an automated fashion. Lucie allows fully automated (re)installation and configuration of every node in a cluster in a very rapid fashion.4) Prototyping scalable, secure and self-organizing cluster communication. We have identified that scalable, reliable, secure, and self-organizing communication within the cluster node is the essential foundation for reliable, plug＆play clustering. We have prototyped some of the ideas in the Gfarm (cluster middleware for Petascale Datagrid processing) job manager : there, the self-organizing process ring structure governs all the nodes, and jobs can be started up rapidly in parallel, in a safe secure manner. Less