Elements: Basil: A Tool for Semi-Automatic Containerization, Deployment, and Execution of Scientific Applications on Cloud Computing and Supercomputing Platforms

软件应用程序的“集装箱化”使其经得起未来的考验，有助于其长期保存，使其可跨不同的硬件平台移植，确保可重复的结果，并使其便于传播。Docker和Singularity是两种用于容器化科学应用程序的流行软件技术，并在不同的硬件平台上得到广泛支持。然而，它们的采用涉及陡峭的学习曲线，特别是在开发感兴趣的应用程序的安全和优化映像时。大量的领域科学家和学者通常没有接受过使用Docker和Singularity容器化应用程序的正式培训，并且花费了大量的时间将他们的应用程序移植到不同的云计算和超级计算平台。将具有多个软件依赖性和敏感性的应用程序移植到特定软件版本的过程对于这样的用户可能是特别艰巨的。为了帮助他们，该项目正在开发BASIL -一种用于半自动容器化科学应用程序，框架和工作流的工具。该项目将通过一个门户网站、一个命令行工具和API来提供BASIL。BASIL在具有深刻社会影响的多个领域具有广泛的适用性，如人工智能，药物发现和地震工程。通过保存有价值的遗留软件并使其在未来几年内可用，BASIL将节省软件重写和软件安装的成本和时间，从而为促进社会繁荣做出贡献。该项目将产生关于“集装箱化入门”的教育内容，参与该项目的学生将在超级计算/高性能计算（HPC）和云计算等国家利益领域发展宝贵的技能。BASIL将是同类工具中第一个可以半自动生成安全，优化和值得信赖的容器映像的工具，并提供有关如何在适当的许可证下使用映像的明确信息。优化映像的规则将来自专家知识和最佳实践，例如多阶段构建和重新排序命令的顺序以利用缓存，从而减少构建映像所需的总时间。BASIL工具的用户将以下列形式之一提供构建其应用程序/工作流的方法：（1）Makefiles/CMakefiles，（2）脚本，（3）命令，或（4）使用项目团队提供的模板的具有预定义关键字和符号的文本文件。这些配方将被解析，并生成Dockerfiles或Singularity定义文件。在这个项目中开发的解析器将是该项目的另一个新的贡献。使用生成的Dockerfile或Singularity定义文件，将构建Docker或Singularity镜像。接下来，图像将被扫描以查找任何漏洞，签名，如果用户愿意，可以使用适当的许可证在公共注册中心发布。这些容器镜像可以使用BASIL Web门户进行测试，并可以拉取以在不同的硬件平台上运行或部署。该奖项由高级网络基础设施办公室（Office of Advanced Cyberinfrastructure）颁发，并得到数学和物理科学理事会（Directorate for Mathematical and Physical Sciences）物理学部（Division of Physics）信息前沿物理学（Physics at the Information Frontier）的共同支持。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The "containerization" of software applications future-proofs them, helps in their long-term preservation, makes them portable across different hardware platforms, ensures reproducible results, and makes them convenient to disseminate. Docker and Singularity are two popular software technologies for containerizing scientific applications and are widely supported on different hardware platforms. However, their adoption involves a steep learning curve, especially when it comes to developing secure and optimized images of the applications of interest. A large number of domain-scientists and scholars are usually not formally trained at containerizing their applications with Docker and Singularity, and spend a significant amount of their time in porting their applications to different cloud computing and supercomputing platforms. The process of porting applications having multiple software dependencies and sensitivities to specific software versions can be especially arduous for such users. To assist them, this project is developing BASIL - a tool for semi-automatically containerizing the scientific applications, frameworks, and workflows. This project will deliver BASIL through a web portal, as a command-line tool, and through APIs. BASIL has a broad applicability across multiple domains of deep societal impact such as artificial intelligence, drug discovery, and earthquake engineering. By enabling the preservation of valuable legacy software and making them usable for several years in future, BASIL will save cost and time in software rewriting and software installations, and thus contribute towards advancing the prosperity of the society. The project will result in educational content on “Introduction to Containerization” and students engaged in the project will develop valuable skills in the areas of national interest such as supercomputing/High Performance Computing (HPC) and cloud computing. BASIL will be the first tool of its kind that can semi-automatically generate secure, optimized, and trustworthy container images with clear information on how to use the images under appropriate licenses. The rules for optimizing the images will be derived from expert knowledge and best practices, such as multi-stage builds and reordering the sequencing of commands to take advantage of caching so that the overall time involved in building the images is reduced. Users of the BASIL tool will provide the recipes for building their applications/workflows in one of the following forms (1) Makefiles/CMakefiles, (2) scripts, (3) commands, or (4) a text-file with predefined keywords and notations using templates provided by the project team. These recipes will be parsed, and Dockerfiles or Singularity definition files will be generated. The parser developed in this project will be another novel contribution of the project. Using a generated Dockerfile or Singularity definition file, a Docker or Singularity image will be built. Next, the image will be scanned for any vulnerabilities, signed, and if the user desires, released in public registries with appropriate licenses. These container images can be tested using the BASIL web portal, and can be pulled to run or deploy on diverse hardware platforms. This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Physics at the Information Frontier in the Division of Physics within the Directorate for Mathematical and Physical Sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.