Transforming colors with matrices | lisyarus blog

There’s one trick we use at work, and now I’m using in my current medieval village building game project, which apparently isn’t as well-known as I thought: transforming colors using matrices, interpreting colors as 3D RGB or 4D RGBA vectors. In this article I’ll try to explain how it works and which operations on colors can be represented this way.

Contents

But why?

I mean, if you’re a graphics programmer, you’ve probably worked with colors before, and you probably never needed to apply matrices to colors, unless you’re doing something hardcore like transforming to CIE XYZ or Oklab color spaces. You probably needed to compute lighting, apply some post-processing and tone-mapping, – operations which are almost never linear. Why matrices?

At work, we’re rendering stylized maps, and we allow the user to have a fair amount of customization. In particular, the user can decide to e.g. roads darker, make buildings more saturated, and shift the hue of forests into the yellow region. All these are simple operations, but they are vastly different in implementation. It would be a nightmare to support a dozen uniform parameters in the shader just to implement all these operations.

However, we can represent all these operations as matrices! Now composition of these is trivial (matrix-matrix multiplication), and applying them in the shader is also trivial (matrix-vector multiplication). Matrices are simple, efficient, and easy to reason about, so it’s a total win-win.

In my game project, any type of resource (wood, meat, metal, etc) can have a certain material: e.g. there are currently 4 wood materials, namely birch, poplar, oak, and spruce. This material affects the color of the object via a special per-material color matrix which is applied to the object’s albedo before doing any lighting calculations in the shader.

For this to work, the meshes themselves should have some unusual coloring: in the case of wood materials, which use just two colors (a lighter and a darker one), the raw mesh colors are combinations of red and green which sum to 1 (otherwise the resulting color would overflow):

Pros’n’cons

So, matrices can describe a wide range of color transformations (which we’ll discuss in detail shortly), they are easy to combine and apply. Are there any disadvantages to doing this?

Of course there are, or, rather, there is one main problem: not all operations on colors are linear, so not all of them can be represented as matrices. A few examples:

• Transformation between color spaces (RGB \(\leftrightarrow\) HSV \(\leftrightarrow\) CIE XYZ, etc) is typically non-linear
• Gamma-correction is non-linear
• Tone mapping is non-linear (apart from the trivial “divide by max” tone-mapping that nobody uses)
• Hue shifting is non-linear, if done correctly

etc, etc.

So, if you really need some operations that involve non-linear stuff, you just can’t use matrices for that. Maybe you can figure out some clever way to apply a fixed non-linear transformation to your RGBA vector, then apply the matrix, then apply the inverse of said non-linear transformation to obtain the effects you want, but I won’t be talking about this option.

To premultiply or not to premultiply

Premultiplied alpha is a way of storing semi-transparent color values not as \((R,G,B,A)\) vectors, but instead as \((R\cdot A,G\cdot A,B\cdot A,A)\) vectors. This approach tends to work better in most formulas, behaves nicer with color mixing, image filtering, mipmapping, blurring, and so on. The downsides are that

• You lose some precision in the color channels. E.g. if storing each channel as an unsigned byte, if the value of the alpha channel is, say, 15, you have only 16 (from 0 to 15) possible values for the color channels.
• In particular, for pixels that are completely transparent (alpha = 0), the color information is lost completely.

How bad these problems are is a different question, but note that if a pixel is really close to being completely transparent, you probably won’t notice the severe color quantization in it anyway.

The reason I’m bringing this up is that premultiplied colors work better with matrices! Specifically, note that a premultiplied color is the same as fully-opaque color of the same shade, multiplied by the alpha value:

\[ \begin{pmatrix}R\cdot A \\ G\cdot A \\ B\cdot A \\ A \end{pmatrix} = A \cdot \begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} \]

Now, multiplication by a constant \(A\) is a very nice operation, because it commutes with all linear operations! I.e. if we have a matrix \(M\) and some vector \(V\), it doesn’t matter if we apply the matrix and then multiply by \(A\), or if we first multiply by \(A\) and then apply the matrix – the results are identical:

\[ M \cdot A \cdot V = A \cdot M \cdot V \]

This means that, if we have a matrix \(M\) that encodes some color transformation that works for fully opaque colors, then the same matrix encodes the transformation for semi-transparent colors in premultiplied format, and leaves the alpha channel untouched:

\[ M \cdot \begin{pmatrix}R\cdot A \\ G\cdot A \\ B\cdot A \\ A \end{pmatrix} = M \cdot A \cdot \begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} = A \cdot M \cdot \begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} = A \cdot \begin{pmatrix}R’ \\ G’ \\ B’ \\ 1\end{pmatrix} = \begin{pmatrix}R’\cdot A \\ G’\cdot A \\ B’\cdot A \\ A \end{pmatrix} \]

Thus, for working with color matrices, it is reeeally useful to have premultiplied colors. Though, in what follows, I’ll try to mention which matrices work for non-premultiplied semi-transparent colors as well, and which don’t.

Now, let’s see how we can represent some typical color operations as matrices.

Darkening a color

This is probably the simplest operation. Usually darkening means just making the color closer to zero. Let’s parametrize this with a parameter \(t\): \(t=0\) means no darkening, and \(t=1\) means full darkening. Then, the corresponding operation is

\[ \begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} \mapsto (1-t)\cdot \begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} \]

which is just scaling the color channels by \(1-t\). This can be easily achieved with a matrix:

\[ \begin{pmatrix}1-t & 0 & 0 & 0 \\ 0 & 1-t & 0 & 0 \\ 0 & 0 & 1-t & 0 \\ 0 & 0 & 0 & 1\end{pmatrix} \]

If you’re familiar with affine transformations, you might recognize this as the usual 3D scaling matrix in homogeneous coordinates, because it literally is scaling! This matrix works both for premultiplied colors and for non-premultiplied ones.

Lightening a color

I’ll define lightening as lerping the color towards white \((1,1,1)\). Again, \(t=0\) means no lightening, while \(t=1\) means turning the color into white. This is just a typical lerp formula:

\[ \begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} \mapsto (1-t)\cdot\begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} + t\cdot \begin{pmatrix}1 \\ 1 \\ 1 \\ 1\end{pmatrix} \]

which can be achieved by the matrix

\[ \begin{pmatrix}1-t & 0 & 0 & t \\ 0 & 1-t & 0 & t \\ 0 & 0 & 1-t & t \\ 0 & 0 & 0 & 1 \end{pmatrix} \]

This time the matrix doesn’t work for non-premultiplied colors with \(A\neq 1\), because it uses the 4-th color component to add the value of \(t\) to other channels, similar to how affine translations use the \(W\) component of points in homogeneous coordinates to implement the translation as a \(4\times 4\) matrix.

In fact, the formula for non-premultiplied colors would be

\[ \begin{pmatrix}R \\ G \\ B \\ A\end{pmatrix} \mapsto \begin{pmatrix}(1-t)\cdot R + t \\ (1-t)\cdot G + t \\ (1-t)\cdot B + t \\ A\end{pmatrix} \]

which is not a linear operation, and cannot be represented as a matrix.

Blending with a fixed color

Both darkening and lightening are special cases of lerping towards a fixed color \((R’,G’,B’)\) with a factor of \(t\). Equivalently, this is the same as alpha-blending the color \((R’,G’,B’,t)\) over our color, while leaving the alpha channel of our color unchanged. Again, this is a well-known lerp formula:

\[ \begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} \mapsto (1-t)\cdot\begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} + t\cdot \begin{pmatrix}R’ \\ G’ \\ B’ \\ 1\end{pmatrix} \]

and the matrix is

\[ \begin{pmatrix}1-t & 0 & 0 & t\cdot R’ \\ 0 & 1-t & 0 & t\cdot G’ \\ 0 & 0 & 1-t & t\cdot B’ \\ 0 & 0 & 0 & 1 \end{pmatrix} \]

And again, this doesn’t work for non-premultiplied colors.

Replacing with a fixed color

As a special case of blending with \(t=1\), let’s look at the operation of entirely replacing our color with a new color \((R’,G’,B’)\). Just set \(t=1\) in the previous section to get

\[ \begin{pmatrix}0 & 0 & 0 & R’ \\ 0 & 0 & 0 & G’ \\ 0 & 0 & 0 & B’ \\ 0 & 0 & 0 & 1 \end{pmatrix} \]

This also doesn’t work for non-premultiplied colors.

Adding a fixed color

While this operation is somewhat weird, we can still implement it: add another color \((R’,G’,B’)\) to our color, component-wise. This can, of course, make the color values overflow, but we’ll pretend that the user knows what to do with that. The formula is

\[ \begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} \mapsto \begin{pmatrix}R \\ G \\ B \\ 1\end{pmatrix} + \begin{pmatrix}R’ \\ G’ \\ B’ \\ 0\end{pmatrix} \]

(notice the 0 in the alpha channel of the second vector), and the matrix is

\[ \begin{pmatrix}1 & 0 & 0 & R’ \\ 0 & 1 & 0 & G’ \\ 0 & 0 & 1 & B’ \\ 0 & 0 & 0 & 1 \end{pmatrix} \]

which is just the usual \(4\times 4\) translation matrix in homogeneous coordinates. This also doesn’t work for non-premultiplied colors, and instead adds the color \((R’,G’,B’)\) multiplied by the input color alpha.

By the way, we can, of course, subtract a fixed color instead: just make the \((R’,G’,B’)\) negative.

Changing contrast

Contrast is about the grayness of the color. Color with zero contrast is just the uniform gray \((0.5,0.5,0.5)\). Color with contrast of 1 is our original color. You can probably guess that color with contrast \(t\) is just lerping from our color to the 0.5-gray with a lerping parameter of \(1-t\). Substitute \(t \mapsto 1-t\) and \((R’,G’,B’) = (0.5,0.5,0.5)\) in the blending section to get

\[ \begin{pmatrix}t & 0 & 0 & \frac{1-t}{2} \\ 0 & t & 0 & \frac{1-t}{2} \\ 0 & 0 & t & \frac{1-t}{2} \\ 0 & 0 & 0 & 1 \end{pmatrix} \]

Note that \(t=1\) gives the identity matrix (i.e. the color doesn’t change), while \(t=0\) gives the matrix of replacing with the fixed gray color. Again, this doesn’t work for non-premultiplied colors.

Changing saturation

Saturation is somewhat similar to contrast, but instead of grayness, it is about vividness of the colors. The simplest way to implement saturation is to compute the luminance \(L\) of our color, and then lerp towards/from the gray color \((L,L,L)\). Note that this is similar to contrast, but now the gray point depends on the input color.

Thankfully, all these operations are still linear. Let’s take the luminance formula from wikipedia to get

\[ L = 0.2126\cdot R + 0.7152\cdot G + 0.0722\cdot B = w_R\cdot R + w_G \cdot G + w_B \cdot B \]

where I’ve defined the luminance weights \((w_R, w_G, w_B)\) for convenienve. Now the matrix is

\[ \begin{pmatrix}t & 0 & 0 & (1-t)\cdot L \\ 0 & t & 0 & (1-t)\cdot L \\ 0 & 0 & t & (1-t)\cdot L \\ 0 & 0 & 0 & 1 \end{pmatrix} \]

This matrix still depends on the original color, though (via the \(L\) parameter). Let’s explicitly write the resulting color vector:

\[
\begin{pmatrix}
t\cdot R + (1-t)\cdot L \\
t\cdot G + (1-t)\cdot L \\
t\cdot B + (1-t)\cdot L \\
1
\end{pmatrix}
=
\begin{pmatrix}
t\cdot R + (1-t)\cdot (w_R\cdot R + w_G \cdot G + w_B \cdot B) \\
t\cdot G + (1-t)\cdot (w_R\cdot R + w_G \cdot G + w_B \cdot B) \\
t\cdot B + (1-t)\cdot (w_R\cdot R + w_G \cdot G + w_B \cdot B) \\
1
\end{pmatrix}
= \\
=
\begin{pmatrix}
(t+(1-t)w_R)\cdot R + (1-t)w_G\cdot G + (1-t)\cdot B \\
(1-t)w_R\cdot R + (t+(1-t)w_G)\cdot G + (1-t)\cdot B \\
(1-t)w_R\cdot R + (1-t)w_G\cdot G + (t+(1-t))\cdot B \\
1
\end{pmatrix}
\]

This lets us to see the matrix directly:

\[
\begin{pmatrix}
t+(1-t)w_R & (1-t)w_G & (1-t)w_B & 0 \\
(1-t)w_R & t+(1-t)w_G & (1-t)w_B & 0 \\
(1-t)w_R & (1-t)w_G & t+(1-t)w_B & 0 \\
0 & 0 & 0 & 1
\end{pmatrix}
\]

In this case, since the \(t=0\) color depends linearly on the input color, the whole operation still works for non-premultiplied colors (as opposed to changing contrast, where the \(t=0\) color is fixed).

Permuting channels

Say, I want to implement a very simple hue shift by interchanging the red and green channels. Easy:

\[ \begin{pmatrix}0 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{pmatrix} \]

This does work for non-premultiplied colors, since it doesn’t involve translating the color, i.e. adding or subtracting something to/from it.

Hue shifting

Ok, now I want to implement a proper hue shifting by an arbitrary angle for my color. Bad news: the proper way of doing this (converting to HSV, shifting the hue, converting back to RGB) is not linear. However, we can cheat a little, and use a simpler formula.

Imagine an RGB color cube: the X axis corresponds to the red channel, and so on. Rotating the hue means rotating this cube around its diagonal: rotating by exactly \(120^\circ\) will turn red into green, green into blue, and blue into red. However, if we rotate by an angle that is not a multiple of \(120^\circ\), parts of the cube will end up somewhere outside the cube. This is the cheating I was talking about: we will pretend that this is not an issue, and let the user deal with the resulting color overflow and clipping.

Now, the matrix of this operation is just the matrix of rotation around the cube diagonal by some angle \(\theta\). The diagonal is \((1,1,1)\), or, since the rotation axis is assumed to be normalized, it should be \(\left(\frac{1}{\sqrt 3}, \frac{1}{\sqrt 3}, \frac{1}{\sqrt 3}\right)\). Grab the matrix directly from wikipedia and we get

\[
\begin{pmatrix}
\frac{1 + 2\cos\theta}{3} & \frac{1 – \cos\theta}{3} – \frac{\sin\theta}{\sqrt 3} & \frac{1 – \cos\theta}{3} + \frac{\sin\theta}{\sqrt 3} & 0 \\
\frac{1 – \cos\theta}{3} + \frac{\sin\theta}{\sqrt 3} & \frac{1 + 2\cos\theta}{3} & \frac{1 – \cos\theta}{3} – \frac{\sin\theta}{\sqrt 3} & 0 \\
\frac{1 – \cos\theta}{3} – \frac{\sin\theta}{\sqrt 3} & \frac{1 – \cos\theta}{3} + \frac{\sin\theta}{\sqrt 3} & \frac{1 + 2\cos\theta}{3} & 0 \\
0 & 0 & 0 & 1
\end{pmatrix}
\]

Not the prettiest thing ever, but hey, at least that’s a matrix. Also, if you already have a decent maths library around, you probably already have a function that computes this matrix, so just call it with \(\text{axis}=\left(\frac{1}{\sqrt 3}, \frac{1}{\sqrt 3}, \frac{1}{\sqrt 3}\right)\) and \(\text{angle}=\theta\).

This one also works with non-premultiplied colors: it just rotates the \((R,G,B)\) part around the origin, without needing to translate the color anywhere.

Tweaking the alpha channel

If I want to, say, make the color 50% more transparent, this is easy: just multiply the alpha channel by \(0.5\). For premultiplied colors, the matrix would be

\[ \begin{pmatrix} 0.5 & 0 & 0 & 0 \\ 0 & 0.5 & 0 & 0 \\ 0 & 0 & 0.5 & 0 \\ 0 & 0 & 0 & 0.5 \end{pmatrix} \]

and for non-premultiplied, it is simply

\[ \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0.5 \end{pmatrix} \]

Unfortunately, if I want to make the color 50% more opaque, i.e. replace the alpha channel \(A\) by \(\frac{1-A}{2}\), this wouldn’t work, regardless of whether the color is premultiplied or not. To make things like lerping possible for opaque colors, we used the \(A=1\) to make translation into a linear operation, much like \(4×4\) affine translation matrix uses the forth \(W=1\) coordinate to do the same. For premultiplied colors it worked automatically, as we’ve discussed in the beginning of the article.

However, to add something to the alpha channel, we’d need some other (fifth?) channel equal to 1, which we could use to add something to our channel. Typically colors have 3 or 4 channels, though, and never 5. So, we cannot represent this operation as a \(4\times 4\) RGBA matrix, and similarily most non-trivial operations with the alpha channel as well.

Everything in one table

Here I’ve organized all operations mentioned above in a single table. All these work with opaque or premultiplied colors:

Hope this will be useful for you! Cheers, and thanks for reading.