Matrix Multiplication: Visualising the Computation Process

To multiply matrix A by matrix B, producing C, we have to do a number of things:

1. Ensure A has the same number of columns as B has rows
2. For each row in A, take its dot product with each column in B
3. Remember where the output of those dot products should be placed in C

While none of these steps are difficult, the process as a whole isn't easy to remember – we have to think horizontally for A, vertically for B, remember to use the dot product, and then remember where the output should be placed in C. This is error-prone and can lead us to focus solely on the computation, losing sight of the more important part: what the result means.

With that in mind, I'm going to present the matrix multiplication process visually in way that I hope is more intuitive and easier to recall. The same visualisation can be applied to any combination of vectors and matrices.

A visual approach

Allow me to introduce you to Dotty. Dotty is a machine that has one function: perform the dot product on two sets of input and spit out the result. The following animation demonstrates using Dotty to multiply a 3x3 matrix (red) by a 3x2 matrix (green and blue) to produce a 3x2 matrix (gold and purple). This is a typical example you might find written as AB = C in a textbook, with A being red, B being green and blue, and C being gold and purple. Click anywhere in the box below to play the animation:

The key part is sliding the green and blue matrix up and tipping it on its left side. This makes the whole process easier to remember as we no longer have to think horizontally for A's rows and vertically for B's columns. We can now imagine green sliding down through the machine and Dotty calculating the dot product of green with each of red's rows in turn to produce each element in gold – the first row produces the first element in gold, the second row produces the second element in gold, and so on.

Green was the first portion of B that we put through Dotty, and it produced the first column of C. Once we're finished with green, we repeat the process for blue by sliding it down through Dotty. As blue is the second portion of B, running it through Dotty produces the second column of B, which is purple.

Visually, I think this makes a lot of sense. But there is more information we can get from this, which answers the question: which matrix/vector combinations can be multiplied? Well, if we do as before and take B, push it up, tip it on its left side, does it fit exactly on A? If there are too few or too many elements, then we can't multiply it with A. Here is an example:

The animation shows the number of columns in red doesn't match the number of elements in green. Even if we perform the dot product on the first two elements, it's not clear what we would do with the leftover green element. As such, this multiplication isn't defined. Similar reasoning can be applied when there are fewer elements in green than in a red row:

To see that this works for vectors too, let's take the case of multiplying a row vector by a column vector:

Finally, visualising with Dotty also demonstrates matrix multiplication is not commutative. As an exercise, try it yourself: visualise multiplying a 2x2 matrix, A by a 2x3 matrix, B. If we slide B up and turn on its side, we can see AB is defined. But if we try the same with BA, sliding A up and tipping it on its side so it's sitting on top of B, we can see BA isn't defined. If you're not convinced that BA isn't defined, take a look at the third example on this page.

So the next time you need to perform matrix multiplication by hand, imagine taking the second matrix (or vector), moving it up and tipping it on its left side, then running it through Dotty. Hopefully you'll find this method easier to remember than the methods described in textbooks.