Efficient Cross-Domain DSL Development for Exascale

开发科学软件，例如用于气候建模或医学研究，是一项极具挑战性的任务。领域科学家经常深入研究底层编程细节，只是为了让他们的代码运行得足够快。这些繁琐但重要的优化步骤大大降低了领域科学家的工作效率。领域特定语言（DSL）使领域科学家能够专注于科学问题，而不是让代码快速运行，从而彻底改变了领域科学家的工作效率。复杂的DSL编译器自动从特定领域的高级问题描述生成高性能代码。虽然有个别成功，但DSL的现有景观是分散的，并且DSL编译器实现中软件组件的重用受到限制，因为传统的DSL编译器是孤立构建的。这导致了新DSL的高开发成本，并阻止了许多DSL达到成熟和可持续的水平，从而使科学界能够采用。该项目通过利用MLIR编译器和Python生态系统的广度和跨行业支持，彻底改变了DSL编译器实现的设计。Python是许多领域的应用程序开发人员的首选工具，例如机器学习，数据科学，以及-我们相信-未来高性能计算软件的重要组成部分。该项目将MLIR作为DSL编译器开发中多个抽象级别的代码的公共表示。嵌入各种宿主语言（包括Python和Fortran）的DSL将很容易在MLIR之上构建。我们的研究将构建一个工具箱，使开发人员能够使用共享中间表示IR和优化的丰富生态系统来构建DSL，而不是将DSL编译器构建为孤立的单片塔。该项目评估，驱动和演示DSL设计工具箱，以构建下一代地震和气候建模以及医学成像的DSL。它们将共享共同的软件组件，并使它们可用于其他DSL。最后，本项目将探讨未来的颠覆者（包括人工智能、数据科学和按需HPC即服务）将如何塑造和影响下一代高性能软件。该项目将致力于将Python生态系统中的现代交互式数据分析和机器学习方法与高性能科学代码进行深度整合。

Developing scientific software, for example for climate modeling or medical research, is a highly challenging task. Domain scientists are often deeply involved in low-level programming details just to make their code run sufficiently fast. These tedious, but important, optimization steps significantly reduce the productivity of scientists.Domain specific languages (DSLs) revolutionize the productivity of domain scientists by enabling them to focus on scientific questions rather than making their code run fast. Sophisticated DSL compilers automatically generate high-performance code from domain-specific high-level problem descriptions.While there are individual successes, the existing landscape of DSLs is scattered and the reuse of software components in DSL compiler implementations is limited as traditionally DSL compilers are built in isolation. This results in high development costs of new DSLs and prevents many DSLs from ever achieving a level of maturity and sustainability that enables uptake by the scientific community.This project revolutionizes the design of DSL compiler implementations by leveraging the breadth and cross-industry support of the MLIR compiler and Python ecosystems. Python is the tool of choice for application developers in many domains, such as machine learning, data science, and - we believe - an important component of the future of High Performance Computing software. This project establishes MLIR as a common representation for code at multiple levels of abstraction in DSL compiler development. DSLs embedded in various host languages, including Python and Fortran, will be easily built on top of MLIR. Instead of building DSL compilers as isolated monolithic towers, our research will build a toolbox that enables developers to build DSLs using a rich ecosystem of shared intermediate representations IRs and optimizations.This project evaluates, drives, and demonstrates the DSL design toolbox to build the next generation of DSLs for Seismic and Climate Modelling as well as Medical imaging. These will share common software components and make them available for other DSLs. An extensive evaluation will show the scalability of DSL software towards exascale.Finally, this project investigates how future disruptors, including artificial intelligence, data science, and on-demand HPC-as-a-service, will shape and influence the next generations of high performance software. This project will work towards deeply integrating modern interactive data analytics and machine learning methods from the Python ecosystem with high-performance scientific code.