Can your face model handle masks prevalent due to COVID-19?

Yesterday, I was talking to a friend regarding developing a model that could detect masks on faces. Based on some insights she gave me, I started looking for datasets with masks. But then a question struck my mind!

Will we have to update all our face models to accommodate face detection with masks on?

I was intrigued to find the answer and honestly expected that my previous model would probably fail at this. Let’s see what I found in my research.

iPhone Face ID

If you have one of the latest iPhones that has a Face ID detection system to unlock your phone, you would be having trouble just like me while wearing a mask. Because most of the face is covered, the system is unable to decipher the exact face and hence, fails to unlock the device.

I try to go out for a walk and have to keep track of the time, next song to be played and steps taken. I have to constantly keep unlocking my device to check these stats. This gets very annoying as I always have a mask on and it takes an extra 5–10 seconds for it to ask for a passcode rather than Face ID and then for me to type it in.

Apple recently released a new iOS version 13.5 which detects a face with a mask and directly prompts the passcode option, which makes it super easy and saves time too.While this is great, the original system is still broken as I can no longer use Face ID with my mask. It’s understandable that it cannot rely on just half of the face but it is just an extra overhead that we’d like to avoid.

Security triumphs always!

Face detection model

While systems such as Face ID cannot compromise security, what about other models that just detect face and are not intended for security. One possible scenario could be counting the number of people (faces) in a gathering to get a rough estimate of the people that showed up.

To test this, I decided to use a model that I had used previously for detecting faces. The code is available below:

Honestly, I expected that the model is also likely to fail given that most of the face covered with a mask, doesn’t clearly resemble a face. Furthermore, the model is not trained on faces with masks.

However, the results are surprising.

The model was accurately able to detect faces even when they had their masks on and that too with a very high confidence.

I can think of two reasons why the model worked:

• Firstly, the model analyzes the change in pixel values to detect the face. Even with the mask on, the change in intensity of pixel values between background and face is noticeable.
• The general structure of the face is still the same even when the mask is on the face and hence, the model is probably able to identify the face.

Below are a few images I tested. All images are taken from https://unsplash.com/

Also note the face detector file expects a PNG file, so I had to update the code such that it accepts a JPG file.

Images with no face masks

Notice how the model is performing exceptionally well with faces without masks. A confidence value of above 90% for all images.

Images with face masks

We notice that the model still identifies the faces correctly and for the most part, the confidence is pretty high too (not 100% in any case).

Conclusion

Contrary to what I originally expected, not all models need to be updated. We need to carefully identify all the use cases where face detection models are currently used and might need re-training.

We note that face detection models may need to be updated to accommodate the new trends introduced by COVID-19, which is the frequent use of masks when going out or in gatherings.