SoD: Algorithmic Differentiation of Functional Programs

点击翻译按钮获取中文摘要

CCF-0438806Jeffrey Mark SiskindPurdue UniversityAlgorithmic Differentiation of Functional ProgramsAutomatic Differentiation (AD) is an established enterprise that seeks to take the derivatives of functions specified as imperative programs through symbolic manipulation rather than finite differencing. A framework for applying AD techniques to modern functional-programming languages, instead of imperativeprograms, will be investigated. This framework will treat AD operators as higher-order functions that map function objects to function objects. These operators will be compositional and will apply to more general functions than the maps between real vectors to which traditional AD transformations are limited. This framework offers several advantages:- a callee can perform the necessary AD rather than insisting that callers provide appropriately transformed functions, making it practical to compositionally build an efficient library of functions based on derivatives - higher-order derivatives can be constructed naturally, without special mechanisms- gradients can be taken through processes that themselves involve AD-based optimization or approximate iterate-to-fixedpoint operators such as PDE solversThe proposed work will have broad impact in such diverse fields as engineering design, scientific computing, and machine learning, where researchers regularly need to take derivatives of complex programs. It has intellectual merit as it offers a novel synthesis of the differential calculus with the lambda calculus.