Algorithmic Differentiation of Numerical Methods: Second-Order Adjoint Solvers for Parameterized Systems of Nonlinear Equations
Algorithmic Differentiation of Numerical Methods: Second-Order Adjoint Solvers for Parameterized Systems of Nonlinear Equations
复制标题
数值方法的算法微分：非线性方程参数化系统的二阶伴随求解器
DOI：
10.1016/j.procs.2016.05.388
复制
发表时间：
2016
影响因子：
1.6
通讯作者：
U. Naumann
中科院分区：
文献类型：
--
作者：
Niloofar Safiran;J. Lotz;U. Naumann
伴随模式算法（也称为自动）微分（AD）将多变量向量函数的计算机程序实现转换为一阶伴随代码。它的重新应用或与切模AD的组合产生高阶伴随码。二阶导数在非线性规划中起着重要的作用。例如，二阶（牛顿型）非线性优化方法通过考虑二阶导数信息，保证在最小值附近的更快收敛。伴随模式是特别感兴趣的大规模基于梯度的非线性优化，由于其计算成本上的自由变量的数量的独立性。目标函数的一部分可以隐式地表示为非参数化非线性方程组的解。如果系统参数依赖于目标的自由变量，则需要非线性系统解关于这些参数的二阶导数。局部计算开销以及AD计算解向量相对于参数的二阶伴随的额外存储器需求取决于非线性求解器执行的迭代次数。这种依赖性可以通过对非线性系统进行符号微分来消除。
Adjoint mode algorithmic (also know as automatic) differentiation (AD) transforms implementations of multivariate vector functions as computer programs into first-order adjoint code. Its reapplication or combinations with tangent mode AD yields higher-order adjoint code. Second derivatives play an important role in nonlinear programming. For example, second-order (Newton-type) nonlinear optimization methods promise faster convergence in the neighborhood of the minimum through taking into account second derivative information. The adjoint mode is of particular interest in large-scale gradient-based nonlinear optimization due to the independence of its computational cost on the number of free variables. Part of the objective function may be given implicitly as the solution of a system ofnparameterized nonlinear equations. If the system parameters depend on the free variables of the objective, then second derivatives of the nonlinear system's solution with respect to those parameters are required. The local computational overhead as well as the additional memory requirement for the computation of second-order adjoints of the solution vector with respect to the parameters by AD depends on the number of iterations performed by the nonlinear solver. This dependence can be eliminated by taking a symbolic approach to the differentiation of the nonlinear system.