EMS®: A Massive Computational Experiment Management System towards Data-driven Robotics
EMS®: A Massive Computational Experiment Management System towards Data-driven Robotics
复制标题
DOI：
10.1109/icra48891.2023.10160948
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
Qinjie Lin;Guo Ye;Han Liu
中科院分区：
文献类型：
--
作者：
Qinjie Lin;Guo Ye;Han Liu
我们提出了EMS®，一个支持高通量计算机器人研究的云支持的大规模计算实验管理系统。与现有系统相比，EMS®具有基于天空的管道编排器，使我们能够轻松地利用异构计算环境（例如，本地集群、公共云、边缘设备）来最佳地部署大规模计算作业（例如，具有超过百万的计算小时）。本文以这种基于天空的流水线编排器为基础，介绍了EMS®软件体系结构的三个抽象层：（i）配置管理层，侧重于自动枚举实验配置;（ii）依赖性管理层，侧重于管理每个实验配置内的复杂任务依赖性;（iii）计算管理层，其关注于使用给定计算资源最优地执行计算任务。这样的架构设计大大提高了数据驱动机器人研究的可扩展性和可重复性，从而大大提高了生产力。为了证明这一点，我们将EMS®与更传统的方法在训练移动的机器人的离线强化学习问题上进行了比较。我们的研究结果表明，EMS®在两个数量级上优于传统方法（在实验高吞吐量和成本方面），只需更改几行代码。我们还利用EMS®开发移动的机器人、机器人手臂和双足应用，证明其适用于许多机器人应用。
We propose EMS®, a cloud-enabled massive computational experiment management system supporting high-throughput computational robotics research. Compared to existing systems, EMS® features a sky-based pipeline orchestrator which allows us to exploit heterogeneous computing environments painlessly (e.g., on-premise clusters, public clouds, edge devices) to optimally deploy large-scale computational jobs (e.g., with more than millions of computational hours) in an integrated fashion. Cornerstoned on this sky-based pipeline orchestrator, this paper introduces three abstraction layers of the EMS® software architecture: (i) Configuration management layer focusing on automatically enumerating experimental configurations; (ii) Dependency management layer focusing on managing the complex task dependencies within each experimental configuration; (iii) Computation management layer focusing on optimally executing the computational tasks using the given computing resource. Such an architectural design greatly increases the scalability and reproducibility of data-driven robotics research leading to much-improved productivity. To demonstrate this point, we compare EMS® with more traditional approaches on an offline reinforcement learning problem for training mobile robots. Our results show that EMS® outperforms more traditional approaches in two magnitudes of orders (in terms of experimental high throughput and cost) with only several lines of code change. We also exploit EMS® to develop mobile robot, robot arm, and bipedal applications, demonstrating its applicability to numerous robot applications.