SoD: Algorithmic Differentiation of Functional Programs

jeffrey Mark siskin普渡大学函数程序的算法微分自动微分（AD）是一个成熟的企业，旨在通过符号操作而不是有限差分来获取指定为命令式程序的函数的导数。我们将研究将AD技术应用于现代函数式编程语言的框架，而不是命令式程序。该框架将AD操作符视为将函数对象映射到函数对象的高阶函数。这些运算符将是组合的，将比传统AD变换所局限的真实向量之间的映射应用于更一般的函数。该框架提供了几个优点：-被调用方可以执行必要的AD，而不是坚持调用方提供适当转换的函数，使得基于导数组合构建高效的函数库变得可行-可以自然地构造高阶导数；在没有特殊机制的情况下，梯度可以通过本身涉及基于ad的优化或近似迭代到不动点算子（如PDE解算器）的过程来实现。提议的工作将在工程设计，科学计算和机器学习等不同领域产生广泛的影响，研究人员经常需要对复杂程序进行导数。它具有智力上的价值，因为它提供了一种新的微分与微积分的综合。

CCF-0438806Jeffrey Mark SiskindPurdue UniversityAlgorithmic Differentiation of Functional ProgramsAutomatic Differentiation (AD) is an established enterprise that seeks to take the derivatives of functions specified as imperative programs through symbolic manipulation rather than finite differencing. A framework for applying AD techniques to modern functional-programming languages, instead of imperativeprograms, will be investigated. This framework will treat AD operators as higher-order functions that map function objects to function objects. These operators will be compositional and will apply to more general functions than the maps between real vectors to which traditional AD transformations are limited. This framework offers several advantages:- a callee can perform the necessary AD rather than insisting that callers provide appropriately transformed functions, making it practical to compositionally build an efficient library of functions based on derivatives - higher-order derivatives can be constructed naturally, without special mechanisms- gradients can be taken through processes that themselves involve AD-based optimization or approximate iterate-to-fixedpoint operators such as PDE solversThe proposed work will have broad impact in such diverse fields as engineering design, scientific computing, and machine learning, where researchers regularly need to take derivatives of complex programs. It has intellectual merit as it offers a novel synthesis of the differential calculus with the lambda calculus.