Fast Reverse-Mode Automatic Differentiation using Expression Templates in C++
Fast Reverse-Mode Automatic Differentiation using Expression Templates in C++
复制标题
DOI：
10.1145/2560359
复制
发表时间：
2014-06-01
影响因子：
2.7
通讯作者：
Hogan, Robin J.
中科院分区：
文献类型：
--
作者：
Hogan, Robin J.
在许多领域都会遇到基于一致性的优化问题，但区分大型计算机算法的相关任务可能是艰巨的。执行反向模式自动微分的运算符过载方法对用户来说是最方便的，但当前的实现通常比原始算法慢10-35倍。本文提出了一种快速的新的运算符重载方法，它使用C++中的表达式模板编程技术来提供每个数学表达式的编译时表示，作为一个计算图，可以有效地在任何一个方向上遍历。用四种不同的数值算法进行基准测试表明，这种方法比当前的运算符重载库快2.6-9倍，内存使用效率高1.3-7.7倍。它通常小于原始算法的计算成本的4倍，尽管在不包含数学函数的简单循环的情况下，所有库的性能都较差。在Adept C++软件库中可以免费获得一个实现。
Gradient-based optimization problems are encountered in many fields, but the associated task of differentiating large computer algorithms can be formidable. The operator-overloading approach to performing reverse-mode automatic differentiation is the most convenient for the user but current implementations are typically 10-35 times slower than the original algorithm. In this paper a fast new operator-overloading method is presented that uses the expression template programming technique in C++ to provide a compile-time representation of each mathematical expression as a computational graph that can be efficiently traversed in either direction. Benchmarking with four different numerical algorithms shows this approach to be 2.6-9 times faster than current operator-overloading libraries, and 1.3-7.7 times more efficient in memory usage. It is typically less than 4 times the computational cost of the original algorithm, although poorer performance is found for all libraries in the case of simple loops containing no mathematical functions. An implementation is freely available in the Adept C++ software library.