ELEMENTS: CLAD ENABLING DIFFERENTIABLE PROGRAMMING IN SCIENCE

科学家们面临着迅速增长的数据规模和复杂性。深度学习已被证明是理解大数据集的一种非常有效的方式，它使用依赖于自动微分(AD)的反向传播技术来提供高效且可扩展的基于梯度的优化方法。推广深度学习的努力通过结合广泛的数值计算并允许在基于优化的管道中使用通用数字代码而导致了新兴的可区分编程(DP)范例。诸如参数估计、逆问题和仪器设计之类的任务自然地被表述为服从梯度下降的优化问题。DP范式使研究人员能够利用领域知识，同时使用强大的新技术来增强他们的科学。该项目将使可区分编程技术在大规模科学中的使用成为可能。C和C++是对性能敏感的科学计算的首选语言。然而，C++目前是AD实现的主要挑战，它还没有提供良好的功能覆盖和足够的性能。健壮性、性能、本地语言支持和对现代硬件架构的支持是在复杂的科学流水线中采用的关键。该项目将扩展源代码转换AD工具“CLAD”，以提供广泛的C++语言支持和互操作能力。CLAD深度集成到LLVM编译工具链中，它重用了Clang编译器前端以区分C++构造，其中它可以访问语言细节以支持必要的C++功能。通过使用编译器区分代码，CLAD提供了对区分过程的高级控制。Clad已经是一个非常强大的AD研究软件。该项目将：(A)扩展对C++实体的支持，包括并发原语，以便在尊重高级程序结构的同时轻松利用GPU；(B)促进AD互操作性和与大型科学代码的集成；以及(C)便于在科学上采用可区分的编程。这项提议将把CLAD转变为可持续的网络基础设施，并将吸引来自多个领域的科学家使用CLAD作为DP管道的一部分，以从他们的数据中获得新的见解。这项由高级网络基础设施办公室颁发的奖项由数学和物理科学局内的物理部联合支持。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Scientists face a rapidly growing scale and complexity of data. Deep learning has proven to be a tremendously effective way of understanding large data sets, using backpropagation techniques that rely on automatic differentiation (AD) to provide efficient and scalable gradient-based optimization methods. Efforts to generalize deep learning have resulted in the emerging Differentiable Programming (DP) paradigm by incorporating a wide set of numerical computations and allowing the use of general numerical codes in optimization-based pipelines. Tasks such as parameter estimation, inverse problems, and apparatus design are naturally formulated as optimization problems amenable to gradient descent. The DP paradigm enables researchers to leverage domain knowledge while using powerful new techniques to enhance their science. This project will enable the use of differentiable programming techniques in large-scale science. C and C++ are the languages of choice for performance-sensitive scientific computing. However, C++ is currently a major challenge for AD implementations, which do not yet provide good feature coverage and adequate performance. Robustness, performance, native language support, and support for modern hardware architectures are critical for adoption in complex scientific pipelines.This project will extend the source transformation AD tool “Clad” to provide extensive C++ language support and interoperability capabilities. Deeply integrated into the LLVM compilation toolchain, Clad re-uses the Clang compiler frontend to differentiate C++ constructs, where it has access to the language details to support the necessary C++ features. By using the compiler to differentiate code, Clad provides advanced control over the differentiation process. Clad is already a very capable software for AD research. This project will: (a) extend support for C++ entities, including concurrency primitives to easily take advantage of GPUs while respecting high-level program structure; (b) facilitate AD interoperability and integration with large-scale scientific codes; and (c) ease differentiable programming adoption in science. This proposal will change Clad into sustainable cyberinfrastructure, and will engage scientists from numerous domains to use Clad as part of a DP pipeline to gain new insights from their data.This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Division of Physics within the Directorate for Mathematical and Physical Sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.