Fantastic Adjoints And How To Find Them

1 minute read

This is a very short note about the adjoints. I hope it would be as useful for those feeling a bit confused with this wonderful mathematical concept as it was for me. Although there is not a single word about sensitivity, this very short example can give you a sneek peek.

So, what’s the matter?

I have a problem: I need to find a vector, \(u\), satisfying the following linear algebra problem:

\[\begin{equation} A u = F. \end{equation}\]

Here \(A\) is some matrix (presumably very unpleasant and big) or a matrix operator, \(F\) is called the external forcing, and both \(A\) and \(F\) are given. Naturally, I would invert the matrix and multiply the right hand side by it. The answer would be:

\[\begin{equation} u = A^{-1} F. \end{equation}\]

In practice, finding \(u\) itself is rarely the goal. Some function of \(u\) is almost always much more meaningful. This can be, for instance, a linear combination with another vector, \((l, u)\). Picking a vector of ones \(l = \mathbf{1}\) will give us the average value of \(u\).

The Adjoint

We can take the direct operator and do the complex conjugation and matrix transpose. The new adjoint operator then works this way:

\[\begin{equation} (Ax, y) = (x, A^\dagger y) \end{equation}\]

Now let’s introduce the dual problem, which depends on previously discussed vector \(l\):

\[\begin{equation} A^\dagger v = l. \end{equation}\]

Using the definition of adjoint operator, I can show:

\[\begin{equation} (v, F) = (v, A u) = (A^\dagger v, u) = (l, u). \end{equation}\]

This is a wonderful result. It means, that \((l, u)\) can be found for any \(F\) without recomputation of the very first matrix equation just by taking an inner product of \(v\) with the \(F\) we care at the moment.

The only cost of such computation is to calculate \(v = \left(A^\dagger \right)^{-1} l\) once.

Summing up

A problem of finding the inner product \((u, l)\) for \(u\) being the solution of \(A u = F\), and given \(l\) can be cheaply solved for any forcing term \(F\). First, we need to find the adjoint operator \(A^\dagger\). Second, we solve the dual problem \(A^\dagger v = l \\) for a vector \(v\) once. Third, the target inner product equals to \((v, F)\).

Updated:

You may also enjoy

Demystifying python container types

11 minute read

Writing type annotations in python can be hard. How do I pick the right one with so many different options that look so similar? If a function returns [1, 2, 3], how should I annotate it? Should it be an Iterable, or a MutableSequence? Can it be a Generator? And why not just a list? The short answer is, as always, “it depends”, on what exactly you want to do. Let’s go through some of the python types related to container of elements, discuss how those are related, and where to use them.

Sorry, I still don’t understand what type variance is

13 minute read

I enjoy static typing and use it at work and for personal projects. It makes my life easier, but sometimes the errors that my type checker produces confuse me. For example, I have heard of type covariance and contravariance, but I still can’t quite follow when and why should I care about that? In this post we discuss a couple of realistic scenarios that lead us to the ideas of type variance.

Some lessons from my Ph.D.

6 minute read

I want to share some general ideas that will make your PhD journey more enjoyable, less stressful, and much more productive. In my experience, most of the tips are useful beyond your college years: I still follow them, so practicing them during 3 - 4 years of a PhD can benefit you in the next steps in your career. These ideas won’t work unless they become a habit, but I believe that it is worthwhile trying them out and adjusting to your own needs.