Distributed Asynchronous Array Computing with the JetLag Environment
Distributed Asynchronous Array Computing with the JetLag Environment
复制标题
DOI：
10.1109/pyhpc51966.2020.00011
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Steven R. Brandt;Bita Hasheminezhad;Nanmiao Wu;Sayef Azad Sakin;Alex Bigelow;Katherine E. Isaacs;K. Huck;Hartmut Kaiser
中科院分区：
文献类型：
--
作者：
Steven R. Brandt;Bita Hasheminezhad;Nanmiao Wu;Sayef Azad Sakin;Alex Bigelow;Katherine E. Isaacs;K. Huck;Hartmut Kaiser
我们描述了JetLag，这是一个基于Python的环境，可访问分布式，交互式，异步多任务（AMT）计算框架，称为phylanx。这个环境包括整个计算过程，从用于管理代码和结果的Jupyter前端到性能数据的收集和可视化。我们使用Python Decorator访问Python函数的抽象语法树，然后将它们传输到一组C ++数据结构中，然后由HPX运行时间执行。该环境包括用于发送功能的服务及其在远程资源上的作业的参数。一组Docker和Singularity容器用于简化Jetlag环境的设置。 JetLag系统适用于各种数组计算任务，包括机器学习和探索性数据分析。
We describe JetLag, a Python-based environment that provides access to a distributed, interactive, asynchronous many-task (AMT) computing framework called Phylanx. This environment encompasses the entire computing process, from a Jupyter front-end for managing code and results to the collection and visualization of performance data.We use a Python decorator to access the abstract syntax tree of Python functions and transpile them into a set of C++ data structures which are then executed by the HPX runtime. The environment includes services for sending functions and their arguments to run as jobs on remote resources.A set of Docker and Singularity containers are used to simplify the setup of the JetLag environment. The JetLag system is suitable for a variety of array computational tasks, including machine learning and exploratory data analysis.