``To Be Recorded'' Analysis in Reverse-Mode Automatic Differentiation

Laurent Hascoët Valérie Pascual (INRIA, BP93, 06902 Sophia-Antipolis, France) Uwe Naumann (RWTH Aachen University, 52056 Aachen, Germany)

Article in Future Generation Computer Systems Journal, Elsevier, 2004 (24 pages)

Abstract: The automatic generation of adjoints of mathematical models that are implemented as computer programs is receiving increased attention in the scientific and engineering communities. Reverse-mode automatic differentiation is of particular interest for large-scale optimization problems. It allows the computation of gradients at a small constant multiple of the cost for evaluating the objective function itself, independent of the number of input parameters. Source-to-source transformation tools apply simple differentiation rules to generate adjoint codes based on the adjoint version of every statement. In order to guarantee correctness, certain values that are computed and overwritten in the original program must be made available in the adjoint program. For their datermination we introduce a static data-flow analysis called ``to be recorded" analysis. Possible overestimation of this set must be kept minimal to get efficient adjoint codes. This efficiency is essential for the applicability of source-to-source transformation tools to real-world applications.

Keywords: Automatic Differentiation, Reverse mode, Active variables, Memory requirements, Data-Flow analysis

Full text (pdf)


@article{HNP04,
  author = {Hasco{\"e}t, L. and Naumann, U. and Pascual, V.},
  title = {``To Be Recorded'' Analysis in Reverse-Mode Automatic Differentiation},
  journal = {Future Generation Computer Systems},
  volume = 21,
  number = 8,
  publisher = {Elsevier},
  year = 2004
}