Work Queue + Python: A Framework For Scalable Scientific Ensemble Applications
Work Queue + Python: A Framework For Scalable Scientific Ensemble Applications
复制标题
工作队列 + Python：可扩展科学集成应用程序的框架
DOI：
--
复制
发表时间：
2011
期刊：
影响因子：
--
通讯作者：
D. Thain
中科院分区：
文献类型：
--
作者：
Peter Bui;D. Rajan;Badi Abdul;J. Izaguirre;D. Thain
- 即使随着当今研究科学家可用的计算机资源的数量和种类的增加，构建可扩展的分布式应用程序仍然具有挑战性。为了解决这个问题，我们开发了工作队列，这是一个灵活的主/工人框架，用于构建跨越许多机器（包括集群，网格和云）的大规模科学集成应用程序。在本文中，我们描述了工作队列，然后提出了Python-WorkQueue模块，它使科学家能够利用我们的工作队列框架，同时使用Python编程语言。为了展示模块的可扩展性和功能，我们研究了两个分布式科学应用程序RepExWQ和Folding@work。这两个程序都是使用Python-WorkQueue编写的，并表明工作队列框架不仅能够扩展到数百个工作人员，而且还能够使科学家同时利用多个分布式计算资源。
—Even with the increase in the number and variety of computer resources available to research scientists today, it is still challenging to construct scalable distributed applications. To address this issue, we developed Work Queue, a flexible master/- worker framework for building large scale scientific ensemble applications that span many machines including clusters, grids, and clouds. In this paper, we describe Work Queue and then present the Python-WorkQueue module, which enables scientists to take advantage of our Work Queue framework while using the Python programming language. To demonstrate the module’s flexibility and power, we examine two distributed scientific applications, RepExWQ and Folding@work. Both of these programs were written using Python-WorkQueue and manifest the Work Queue framework’s ability to scale not only to hundreds of workers, but to also enable scientists to take advantage of multiple distributed computing resources simultaneously.