Fantastic Adjoints And How To Find Them

1 minute read

This is a very short note about the adjoints. I hope it would be as useful for those feeling a bit confused with this wonderful mathematical concept as it was for me. Although there is not a single word about sensitivity, this very short example can give you a sneek peek.

So, what’s the matter?

I have a problem: I need to find a vector, \(u\), satisfying the following linear algebra problem:

\[\begin{equation} A u = F. \end{equation}\]

Here \(A\) is some matrix (presumably very unpleasant and big) or a matrix operator, \(F\) is called the external forcing, and both \(A\) and \(F\) are given. Naturally, I would invert the matrix and multiply the right hand side by it. The answer would be:

\[\begin{equation} u = A^{-1} F. \end{equation}\]

In practice, finding \(u\) itself is rarely the goal. Some function of \(u\) is almost always much more meaningful. This can be, for instance, a linear combination with another vector, \((l, u)\). Picking a vector of ones \(l = \mathbf{1}\) will give us the average value of \(u\).

The Adjoint

We can take the direct operator and do the complex conjugation and matrix transpose. The new adjoint operator then works this way:

\[\begin{equation} (Ax, y) = (x, A^\dagger y) \end{equation}\]

Now let’s introduce the dual problem, which depends on previously discussed vector \(l\):

\[\begin{equation} A^\dagger v = l. \end{equation}\]

Using the definition of adjoint operator, I can show:

\[\begin{equation} (v, F) = (v, A u) = (A^\dagger v, u) = (l, u). \end{equation}\]

This is a wonderful result. It means, that \((l, u)\) can be found for any \(F\) without recomputation of the very first matrix equation just by taking an inner product of \(v\) with the \(F\) we care at the moment.

The only cost of such computation is to calculate \(v = \left(A^\dagger \right)^{-1} l\) once.

Summing up

A problem of finding the inner product \((u, l)\) for \(u\) being the solution of \(A u = F\), and given \(l\) can be cheaply solved for any forcing term \(F\). First, we need to find the adjoint operator \(A^\dagger\). Second, we solve the dual problem \(A^\dagger v = l \\) for a vector \(v\) once. Third, the target inner product equals to \((v, F)\).