Toward an Industrial Strength Automatic Differentiation Product

复杂的现实世界现象的数值模拟是几乎所有科学和工程活动的主要挑战，特别是当它不仅需要模拟一个过程（例如穿过汽车或风力涡轮机表面的气流），而且需要优化过程（例如，调整表面的形状以最小化阻力）时。为了有效地解决大问题，模型必须能够获得准确（没有编码、舍入和截断错误）且计算成本低廉的数值导数（输出相对于输入的灵敏度）。目前大多数用户要么提供手工编码的衍生例程，编程和更新繁琐，容易出现编码错误；或者是有限差分，计算成本高，数值不准确。自动微分（AD）是一组对数值建模代码进行机械转换的技术，使其能够以与手工编码导数相同的速度和相同的精度计算模型值及其本身的数值灵敏度。这是通过使用微积分中的链式法则来实现的，但它直接应用于浮点数值，而不是符号表达式。所谓的逆或伴随AD模式可以为评估模型的计算机程序产生一套完整的灵敏度，其计算成本是对程序本身进行一次评估的五倍——即使有数百万个输入变量和灵敏度。然而，要实现这一点，需要使程序向后运行的能力，恢复过程中的所有中间数值，这就需要开发和使用复杂的软件工具。AD技术在建模和优化方面仍未得到广泛应用，这在很大程度上是由于缺乏合适的用户友好的通用商业强度AD工具。当前的AD工具通过操作符重载或源预处理来工作。运算符重载是可靠的，但速度很慢，并且不支持调用专门的数值库函数，例如线性代数例程。源代码预处理工具需要用户对AD有很高的认识，生成的代码不容易让人理解，并且存在内存管理问题。CompAD团队在EPSRC支持下开发的研究编译器是NAG Fortran95编译器的增强版，并且仍然是世界上唯一内置AD支持的工业强度编译器。在现有的高性能编译器中嵌入AD的重载方法提供了两全其美的效果：重载方法的便利性和源预处理的速度。然而，当前的编译器是一种研究工具，需要专家的帮助才能配置和集成应用程序代码。因此，CompAD编译器的研究版本目前的形式不适合大多数潜在的受益者。这个后续的建议将从CompAD编译器中提取AD功能，将其与NAGWare Fortran库集成，并使最终的原型广泛可用。除了为许多用户提供直接的好处外，这个原型将适合系统的市场测试和开发。该原型将用于捕获用户需求，并为随后的商业上可行的软件AD产品的开发奠定基础，该产品将在中等规模的问题上“开箱即用”。

Numerical simulation of complex real-world phenomena is a major challenge for almost all activities in science and engineering, particularly when it is desired not merely to model a process (such as airflow across the surface of a car or a wind turbine) but to optimise the process (for example, to adjust the shape of the surface so as to minimize the drag). To do this efficiently for large problems, it is essential for the model to have access to numerical derivatives (sensitivities of the outputs with respect to inputs) that are accurate (free from coding, rounding and trunction errors) and computationally cheap. At present most users either provide hand-coded derivative routines, which are tedious to program and update, and prone to coding errors; or finite differences, which are expensive to compute and numerically inaccurate. Automatic Differentiation (AD) is a set of techniques for transforming numerical modelling code mechanically so that it calculates the numerical sensitivities of the model values as well as the values themselves, at the same order of speed as hand-coded derivatives, and to the same precision. This is done by using the chain rule from calculus, but applied directly to floating point numerical values, rather than to symbolic expressions. The so-called reverse, or adjoint, mode of AD can produce a complete set of sensitivities for a computer program evaluating a model, at a computational cost of order five times that of a single evaluation of the program itself - even if there are millions of input variables and sensitivities. However, achieving this requires the ability to make the program run backwards, recovering all the intermediate numerical values in the process, and it is this which requires the development and use of sophisticated software tools. AD techniques are still not widely used in modelling and optimisation, due in large part to a lack of suitable user-friendly general-purpose commercial-strength AD tools. Current AD tools work either by operator overloading or by source pre-processing. Operator overloading is reliable, but slow, and do not provide support for calls to specialist numerical library functions, such as linear algebra routines. Source pre-processing tools require a high user awareness of AD, produce code that is not easy for humans to understand, and have memory management issues.The research compiler developed by the CompAD team with EPSRC support is an enhancement of the NAG Fortran95 compiler, and is still the world's only industrial strength compiler to have built-in support for AD. Embedding an overloading approach to AD within an existing high performance compiler gives the best of both worlds: the convenience of the overloading approach and the speed of source pre-processing. However the current compiler is a research tool, and requires expert assistance to configure and integrate with application code. Consequently the research version of the CompAD compiler is unsuitable in its present form for the majority of potential beneficiaries. This follow-on proposal will extract the AD functionality from the CompAD compiler, integrate it with the NAGWare Fortran Library, and make the resulting prototype widely available. As well as providing an immediate benefit to many users, this prototype will be suitable for systematic market testing and development. The prototype will be used to capture user requirements for, and to underpin subsequent development of, a commercially viable software AD product that will work "out-of-the-box" on problems of moderate size.