## "Optimality"

Music: Lemon Tree, by Fool's Garden
Lyrics: Uwe Naumann

I'm sittin' in front of the computer screen.
Newtons second iteration is what I've just seen.
It's not quite the progress that I would expect
from a code such as mine - no doubt it must be perfect!
Just the facts are not supportive and I wonder...

My linear solver is state of the art.
It does not get better wherever I start.
For differentiation is there anything else?
Perturbing the inputs - can't imagine this fails.
I pick a small epsilon and I wonder...

I wonder how but I still give it a try.
The next change in step size is bound to fly
'cause all I'd like to see is simply optimality.
Epsilon, in fact, appears to be rather small.
A factor of ten should improve it all
'cause all I'd like to see is nearly optimality.

A. D.A.D. A.D.A.D.A. D.A.D. A.D.A.D.A. D.A.D.A.D

A few hours later my talk's getting rude.
The sole thing descending seems to be my mood.
How can guessing the Hessian only take this much time?
N squared function runs appear to be the crime.
The facts support this thesis and I wonder...

Isolation due to KKT
Isolation - why not simply drop feasability?

The guy next door's been saying again and again:
An adjoint Lagrangian might relieve my pain.
Though I don't quite believe him, I surrender

I wonder how but I still give it a try:
Still all I'd like to see is simply optimality.
Epsilon itself has finally disappeared.
Reverse mode AD works no matter how weird,
and I'm about to see local optimality

Yes, I wonder, I wonder...

I wonder how but I still give it a try:
Still all I'd like to see...
I really need to see...
Now I can finally see my cherished optimality :-))))

Music: Angie, by The Rolling Stones
Lyrics: Andreas Griewank

Categorically we state
Never ever do truncate
Cancellation won't arise

When you preprocess the code
Slopes and adjoints will be fine

AD, who can claim that it's his baby
IBM and Microsoft are fools
There's no intellectual property
Just old differentiation rules

Though there' many reals to keep
Try checkpointing otherwise

Music: Sunny, by Cher
Lyrics: Andreas Griewank

AD, hand coding drove everyone insane
Now the tedium is gone
Differentiation is fun
Monster codes here we come
Soon the magic is done
AD, it is true, I love you