Overcoming Challenges to Continuous Integration in HPC
Overcoming Challenges to Continuous Integration in HPC
复制标题
克服 HPC 中持续集成的挑战
DOI：
10.1109/mcse.2023.3263458
复制
发表时间：
2022
影响因子：
2.1
通讯作者：
Katz, Daniel S.
中科院分区：
文献类型：
--
作者：
Gamblin, Todd;Katz, Daniel S.
持续集成（CI）已成为现代软件开发中普遍存在的实践，主要的代码托管服务在流行的平台上提供免费的自动化。 CI 具有重大优势，因为它可以在提交更改之前检测代码中的错误。虽然高性能计算 (HPC) 研究在很大程度上依赖于软件，但 HPC 机器并不被视为“通用”平台。这带来了一些阻碍在 HPC 环境中采用 CI 的挑战，使得难以维护无错误的 HPC 项目，并对研究社区造成不利影响。在本文中，我们探讨了阻碍 HPC CI 的挑战，例如硬件多样性、安全性、隔离、管理策略以及非标准身份验证、环境和作业提交机制。我们提出了多种可以提高 HPC 软件质量和开发人员体验的解决方案。实施这些解决方案需要 HPC 中心进行重大改变，但如果做出这些改变，最终将实现更快、更好的科学。
Continuous integration (CI) has become a ubiquitous practice in modern software development, with major code hosting services offering free automation on popular platforms. CI offers major benefits, as it enables detecting bugs in code prior to committing changes. While high-performance computing (HPC) research relies heavily on software, HPC machines are not considered “common” platforms. This presents several challenges that hinder the adoption of CI in HPC environments, making it difficult to maintain bug-free HPC projects, and resulting in adverse effects on the research community. In this article, we explore the challenges that impede HPC CI, such as hardware diversity, security, isolation, administrative policies, and nonstandard authentication, environments, and job submission mechanisms. We propose several solutions that could enhance the quality of HPC software and the experience of developers. Implementing these solutions would require significant changes at HPC centers, but if these changes are made, it would ultimately enable faster and better science.