Cheaper Adjoints by Reversing Address Computations

Laurent Hascoët Jean Utke Uwe Naumann

Article in Scientific Programming (18 pages)

Abstract: The reverse mode of Automatic Differentiation is widely used in science and engineering. A severe bottleneck for the performance of the reverse mode, however, is the necessity to recover certain intermediate values of the program in reverse order. Among these values are computed addresses, which traditionally are recovered through forward recomputation and storage in memory. We propose an alternative approach for recovery that uses inverse computation based on dependency information. Address storage constitutes a significant portion of the overall storage requirements. An example illustrates substantial gains that the proposed approach yields and we show use cases in practical applications

Keywords: program transformation, automatic differentiation, inverse computation

Full text (pdf)


@article{Hascoet2008Cab,
  author = {Hasco{\"e}t, L. and Utke, J. and Naumann, U.},
  title = {Cheaper Adjoints by Reversing Address Computations},
  journal = {Scientific Programming},
  publisher = "IOS Press",
  Volume = 16,
  Number = {1},
  pages = "81--92",
  year = 2008
}