Research on Programming Languages and Softwares for Automatic Calculation of Partial Derivatives and Rounding Error Estimates

本研究的目的是研究由首席研究员M. Iri提出的一种新方法，该方法可以快速准确地评估函数的偏导数，并在函数的计算值中进行舍入估计。该方法解决了电路和系统分析中计算偏导数和范数时的符号微分、数值微分和舍入误差估计困难。我们称这种方法为“FAD：快速自动微分”，因为我们将这种方法与传统的自动微分相结合。本课题的研究结果如下：我们证实了偏导数实际上可以快速准确地计算，并引入了关于舍入误差估计的加权范数的新概念；2 .提出了一种快速计算向量值函数与给定向量的雅可比矩阵积的新方法；3 .提出了一种利用计算图计算标量函数的Hessian矩阵与给定向量乘积的新方法；4 .我们从“计算子图”的新概念严格描述了新方法的算法，并对算法中出现的量进行了解释；5 .我们提出了一种计算舍入误差精确估计的新算法，并与传统算法进行了比较；我们定义了“可约子图”的新概念和提取可约子图的方法；我们完成了FAD的FORTRAN预处理器的实现，并通过大规模问题的数值实验验证了其性能。在数值计算领域，这些结果将使FAD成为一种广泛应用的基础实用技术，有关FAD的技术和理论性质值得进一步深入研究。

The aim of this research is to investigate a new method proposed by M. Iri, head investigator, which can rapidly and accurately evaluate partial derivatives of a function and rounding estimates in the computed values of the function. The new method solved difficulties in symbolic differentiation, numeric differentiation and rounding error estimation when we compute partial derivatives and norms in circuit and system analysis. We call the method "FAD: Fast Automatic Differentiation" because we combine the method with the conventional sutomatic differentiation. The results obtained by this research project are the following:1. We confirmed that partial derivatives can actually be computed rapidly and accurately, and introduced a new concept of weighted norm with respect to rounding error estimates;2. We proposed a new way to fast calculate the product of the Jacobian matrix of a vector-valued function and a given vector;3. We proposed a new way to calculate the product of the Hessian matrix of a scalar-valued function and a given vector using the computational graph;4. We described rigorously the algorithms of the new method in terms of a new concept of "computational subgraph" and gave interpretation to the quantities appearing in the algorithm;5. We proposed a new algorithm for calculating rigorous and sharp estimates of rounding errors and compared it with the conventional;6. We defined a new concept of "reducible subgraph" as well as a method to extract reducible subgraphs;7. We completed the implementation of a FORTRAN preprocessor for FAD, and confirmed its performance through numerical experiments for large-scale problems.In the field of numerical computation, those results will make FAD a widely usable basic practical technique, and techniques as well as theoretical properties concerning FAD should be worth further defailed investigation.