The Data-Flow Equations of Checkpointing in reverse Automatic Differentiation

Benjamin Dauvergne
Laurent Hascoët
(INRIA, BP93, 06902 Sophia-Antipolis, France)


Proceedings of the International Conference on Computational Science, ICCS 2006, Reading, UK (8 pages)

Abstract: Checkpointing is a technique to reduce the memory consumption of adjoint programs produced by reverse Automatic Differentiation. However, checkpointing also uses a non-negligible memory space for the so-called ``snapshots''. We analyze the data-flow of checkpointing, yielding a precise characterization of all possible memory-optimal options for snapshots. This characterization is formally derived from the structure of checkpoints and from classical data-flow equations. In particular, we select two very different options and study their behavior on a number of real codes. Although no option is uniformly better, the so-called ``lazy-snapshot'' option appears preferable in general.

Keywords: Automatic Differentiation, Adjoint code, Adjoint algorithm, Program reversal, Data-Flow analysis, Reverse Mode

Full text (pdf)

@inproceedings{DauvergneHascoet06,
  author = {Dauvergne, B. and Hasco\"et, L.},
  title = {The Data-Flow Equations of Checkpointing in
        reverse Automatic Differentiation},
  booktitle = {International Conference on
               Computational Science, ICCS 2006, Reading, UK},
  year = 2006
}