ELEMENTS: CLAD ENABLING DIFFERENTIABLE PROGRAMMING IN SCIENCE

科学家们面临着快速增长的数据规模和复杂性。深度学习已被证明是一种非常有效的理解大型数据集的方法，它使用依赖于自动区分（AD）的反向传播技术来提供高效且可扩展的基于梯度的优化方法。推广深度学习的努力导致了新兴的可微分规划（DP）范式，该范式结合了广泛的数值计算集，并允许在基于优化的管道中使用通用数值代码。诸如参数估计、反问题和设备设计等任务自然地被表述为适合梯度下降的优化问题。DP范式使研究人员能够利用领域知识，同时使用强大的新技术来增强他们的科学。这个项目将使在大规模科学中使用可微分编程技术成为可能。C和c++是性能敏感型科学计算的首选语言。然而，c++目前是AD实现的主要挑战，它还没有提供良好的功能覆盖和足够的性能。健壮性、性能、本机语言支持以及对现代硬件架构的支持对于在复杂的科学管道中采用至关重要。该项目将扩展源代码转换AD工具“Clad”，以提供广泛的c++语言支持和互操作性。深度集成到LLVM编译工具链中，Clad重用Clang编译器前端来区分c++结构，在那里它可以访问语言细节来支持必要的c++功能。通过使用编译器来区分代码，Clad提供了对区分过程的高级控制。Clad已经是一个非常强大的广告研究软件。该项目将：(a)扩展对c++实体的支持，包括并发原语，以便在尊重高级程序结构的同时轻松利用gpu；(b)促进AD与大规模科学代码的互操作性和集成；(c)便于在科学领域采用可微规划。该提案将把Clad转变为可持续的网络基础设施，并将吸引来自多个领域的科学家使用Clad作为DP管道的一部分，以从他们的数据中获得新的见解。该奖项由先进网络基础设施办公室颁发，并得到数学和物理科学理事会物理部的联合支持。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Scientists face a rapidly growing scale and complexity of data. Deep learning has proven to be a tremendously effective way of understanding large data sets, using backpropagation techniques that rely on automatic differentiation (AD) to provide efficient and scalable gradient-based optimization methods. Efforts to generalize deep learning have resulted in the emerging Differentiable Programming (DP) paradigm by incorporating a wide set of numerical computations and allowing the use of general numerical codes in optimization-based pipelines. Tasks such as parameter estimation, inverse problems, and apparatus design are naturally formulated as optimization problems amenable to gradient descent. The DP paradigm enables researchers to leverage domain knowledge while using powerful new techniques to enhance their science. This project will enable the use of differentiable programming techniques in large-scale science. C and C++ are the languages of choice for performance-sensitive scientific computing. However, C++ is currently a major challenge for AD implementations, which do not yet provide good feature coverage and adequate performance. Robustness, performance, native language support, and support for modern hardware architectures are critical for adoption in complex scientific pipelines.This project will extend the source transformation AD tool “Clad” to provide extensive C++ language support and interoperability capabilities. Deeply integrated into the LLVM compilation toolchain, Clad re-uses the Clang compiler frontend to differentiate C++ constructs, where it has access to the language details to support the necessary C++ features. By using the compiler to differentiate code, Clad provides advanced control over the differentiation process. Clad is already a very capable software for AD research. This project will: (a) extend support for C++ entities, including concurrency primitives to easily take advantage of GPUs while respecting high-level program structure; (b) facilitate AD interoperability and integration with large-scale scientific codes; and (c) ease differentiable programming adoption in science. This proposal will change Clad into sustainable cyberinfrastructure, and will engage scientists from numerous domains to use Clad as part of a DP pipeline to gain new insights from their data.This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Division of Physics within the Directorate for Mathematical and Physical Sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.