Toward Adjoinable MPI

Jean Utke Laurent Hascoët Patrick Heimbach Chris Hill Paul Hovland Uwe Naumann

Proceedings of 10th IEEE International Workshop on Parallel and Distributed Scientific and Engineering, PDSEC'09, May 25-29, 2009, Rome, Italy (8 pages)

Abstract: Automatic differentiation is the primary means of obtaining analytic derivatives from a numerical model given as a computer program. Therefore, it is an essential productivity tool in numerous computational science and engineering domains. Computing gradients with the adjoint (also called reverse) mode via source transformation is a particularly beneficial but also challenging use of automatic differentiation. To date only ad hoc solutions for adjoint differentiation of MPI programs have been available, forcing automatic differentiation tool users to reason about parallel communication dataflow and dependencies and manually develop adjoint communication code. Using the communication graph as a model we characterize the principal problems of adjoining the most frequently used communication idioms. We propose solutions to cover these idioms and consider the consequences for the MPI implementation, the MPI user and MPI-aware program analysis. The MIT general circulation model serves as a use case to illustrate the viability of our approach.

Keywords: MPI, automatic differentiation, source transformation, reverse mode

Full text (pdf)


@inproceedings{Utke2009TAM,
  author = {Utke, J. and Hasco{\"e}t, L. and Heimbach, P. and Hill, C.
            and Hovland, P. and Naumann, U.},
  title = "Toward Adjoinable MPI",
  booktitle = "Proceedings of the 10th IEEE International Workshop on Parallel
            and Distributed Scientific and Engineering, PDSEC'09",
  location = "Rome, Italy",
  year = "2009"
}