Understand and Visualize Color Spaces to Improve Your Machine Learning and Deep Learning Models

Introduction

“Why do we use the RGB color space as a standard in our training models? Sure, it’s the simplest color space because it’s the default color space. But are there other color spaces that may be more suitable? And can it improve our models?”

These questions came to my mind and I absolutely had to find the answers. So I investigated and did some experiments . I would like to share my results with you. 

At first I started by exploring different color spaces that I found inspiring . So in the first part of this article I will introduce you briefly to these color spaces and their possible applications in Machine Learning and Deep Learning .

There is a wide (infinite) number of color spaces, so I made a selection of the most interesting ones for you. 

• RGB – CMYK
• CIE XYZ – CIE L*a*b – CIE L*u*v
• HSV- HSL- HSI
• Y’UV – Y’IQ – YCbCr – YDbDr
• C1C2C3 – I1I2I3
• HED

In the second part of this post, I experienced these color spaces with a same model , in the same configurations . We will see that from one color space to another, the accuracy of our model can go from simple to twice.

RGB — BGR —CMYK

So how is an image in RGB structured? Basically by adding red, green and blue with different “proportions “. But I don’t tell you more than you already know, I think. The more you add the colors, the more you get a lighter color. That is because they emit light (it is this same principle that we can observe by looking very closely to a screen).

This is to be distinguished from primary light-reflecting colors. It is the reverse mechanism, the subtractivity . The more you add the colors together, the darker the color you get. This is the system that is used in printing , the CMYK (Cyan, Magenta, Yellow, and Black).

Then why the RGB? The truth is, there are as many color spaces as you want. We will see how we build them. But the RGB is about simplicity . That’s how our computer hardware is composed.

RGB is the default color space , even in Machine Learning and Deep Learning. But take a look at the alternatives .

CIE XYZ — CIE L*a*b — CIE L*u*v

We saw that the RGB is device-oriented . The International Commission on Illumination, CIE for its French name “Commission Internationale de l’Eclairage” has set standards in colorimetry. It designs more abstract color spaces to break the boundaries of the RGB standard .

The RGB space, encoded on 3 bytes, allows to represent 40% of the colors that the human eye can perceive. This is why the CIE suggests colour spaces to extend the field of possibilities to what man can actually perceive. Hence the color space CIE XYZ . It provides an extension of the boundaries of the color space to contain all the visible . If we simplify it a moment:

• X roughly corresponds to the red stimulation
• Y corresponds more or less to luminance
• Z roughly corresponds to the blue stimulation

Take a look at the schematic and the way we switch from one color space to another and you will understand two key elements :

• Any choice of three “primary” color can only lead to one subset of available colors .
• There is an infinite number of different color spaces with a matrix passage

The CIE XYZ space is an instrumental spac e which serves as a support for other spaces: The CIE L*a*b and the CIE L*u*v will be interesting to deal with because it introduces the notion of lightness .

The eye has 3 distinct cones to detect colors . One for red, one for green, one for blue. But these cones don’t have the same responsiveness . So the perception of colors is different from the real color (speaking in wavelengths). The CIE L*a*b* color space try to distort the CIE XYZ space to better represent color perception for the human eye :

• L* for lightness black → white
• a* to represent the value on an axis green → red ;
• b* to represent the value on an axis blue → yellow .

To train learning models, the CIE L*a*b may be suitable. This can be seen in Wilson Castro’s paper, where they try to classify Cape Gooseberries according to their ripeness. He and his team tried SVM, ANN, DT and KNN. On each of these models, the CIE L*a*b* color space proved to be more effective than the RGB color space.

Finally, the CIE L*u*v* space is another attempt to approach the perception of human eye. It has the advantage of being good at representing images of natural scenes . Color distances are easier to estimate, especially for distances between greens . In M.P. Rico-Fernándeza’s paper where they use a SVM approach to categorize cultivated species , the CIE L*u*v* color space allows greater accuracy .

HSV- HSL- HSI

Other color spaces are based on a psycological approach . This is the case for HSV , HSL and HSI spaces . All are based on the concepts of color psycology , which is the best way to explain what you see :

• Hue : the predominant color
• Saturation : the purity of the color
• Luminance : the brilliance of colour

These color spaces are said to be cylindrical since they are represented by a cylindrical or conical shape around the color hue . All these spaces have the same base: the hue representing the dominant wavelength.