YouTube-Style Face Detect – Crop and Blur Using Python and OpenCV

###Eye cascade xml File
!wget https://raw.githubusercontent.com/opencv/opencv/master/data/haarcascades/haarcascade_eye.xml -P drive/xxx
###Frontal cascade xml File
!wget https://raw.githubusercontent.com/opencv/opencv/master/data/haarcascades/haarcascade_frontalface_default.xml -P drive/xxx

That's cool!✌

2 - Importing the modules

We may need to import OpenCV using cv2 and we are also going to see how to work with remote images that require urlib.request and NumPy for operations.

Note: we also need to import the  cv2_imshow() since the OpenCV imshow() functions cause the jupyter colab to crash.

import cv2
from urllib.request import urlopen
import numpy as np
from google.colab.patches import cv2_imshow

3- Loading the cascade (pre-trained models + remote image)

we are going to load the cascade and remote image using the below code.

cascades_path = 'drive/xxx/'
# Load the cascade
face_cascade = cv2.CascadeClassifier(cascades_path +'haarcascade_frontalface_default.xml')
# Read the input image
req = urlopen('https://tvseriesfinale.com/wp-content/uploads/2014/04/siliconvalley03-590x331.jpg')
arr = np.asarray(bytearray(req.read()), dtype=np.uint8)
img = cv2.imdecode(arr, -1) # 'Load it as it is'
result_image = img.copy()

4 - Convert the image to grayscale

In this step, we are converting our color image (RGB) to Gray-Scale Image and this can be done by using the  cv2.COLOR_BGR2GRAY function which converts the original images to grayscale.

# Convert frame to grayscale:
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

5 - Detect the face object using detect multiscale

detectMultiScale - Detects objects of different sizes in the input image. The detected objects are returned as a list of rectangles and its a part of face_cascade. we are going to use scaleFactor of 1.1 and  minNeighbors = 4 which is the default value.

# Detect faces
faces = face_cascade.detectMultiScale(gray, 1.1, 4)

6. Finally, draw a rectangle around the face and display the result

Since we get the rectangle x,y with height and width from the previous step we are going to loop around to draw for all the identified faces,

# Iterate over each detected face:
for (x, y, w, h) in faces:
        # Draw a rectangle  to see the detected face (debugging purposes):
        cv2.rectangle(img, (x, y), (x + w, y + h), (255, 255, 0), 2)

# Display the resulting frame
cv2_imshow(img)

The complete source code till this will look like this

cascades_path = 'drive/my-drive/'
face_crop = []
# Load the cascade
face_cascade = cv2.CascadeClassifier(cascades_path +'haarcascade_frontalface_default.xml')
# Read the input image
req = urlopen('https://tvseriesfinale.com/wp-content/uploads/2014/04/siliconvalley03-590x331.jpg')
arr = np.asarray(bytearray(req.read()), dtype=np.uint8)
img = cv2.imdecode(arr, -1) # 'Load it as it is'
result_image = img.copy()
#img = cv2.imread('https://intrasee.com/wp-content/uploads/2016/08/silicon-valley-hero.jpg')
# Convert into grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Detect faces
faces = face_cascade.detectMultiScale(gray, 1.1, 4)
print(faces)

# Iterate over each detected face:
for (x, y, w, h) in faces:
    cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)

# Display the resulting frame
cv2_imshow(img)

In the next part of the post, we are going to see how to crop the detected face and display the result for future recognition.

Cropping the detected part

I got this idea while reading checking out the youtube studio blur feature, where it detects all the faces within a video and gives option for blurring the selected face.

So just wanted to build this crop face + select feature(can be passed via Flask API).:sweat_smile:

Since we have already detected the faces in an image we can easily crop or select and store the pixels

First, we will create an empty List to store all the pixels value and next we will select the pixels inside the rectangle.

face_crop = []
for (x, y, w, h) in faces:
    cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)
    face_crop.append(gray[y:y+h, x:x+w])

for face in face_crop:
   cv2_imshow(face)

Result:

you can see that the area differs for each image (rectangle).

Finally, blurring out (Anonymizing) all the detected faces.

To blur out faces in OpenCV we are going to work with gaussian blur

Image blurring is achieved by convolving the image with a low-pass filter kernel. It is useful for removing noise. It actually removes high-frequency content (eg: noise, edges) from the image. So edges are blurred a little bit in this operation (there are also blurring techniques which don't blur the edges).

In this method, instead of a box filter, a Gaussian kernel is used. It is done with the function,  cv.GaussianBlur() . We should specify the width and height of the kernel which should be positive and odd. We also should specify the standard deviation in the X and Y directions, sigmaX and sigmaY respectively. Gaussian blurring is highly effective in removing Gaussian noise from an image.

Since we have already cropped the faces we know the area to be blurred (img[y:y+h, x:x+w]) so we will pick each image to be blurred and apply the gaussian blur and finally merge the blurred part with the original image.

Let see how it works!

# create a sub face from detected faces:
sub_face = img[y:y+h, x:x+w]

# apply a gaussian blur on this new recangle image
sub_face = cv2.GaussianBlur(sub_face,(23, 23), 30)

# merge this blurry rectangle to our final image
result_image[y:y+sub_face.shape[0], x:x+sub_face.shape[1]] = sub_face

# Display the output
cv2_imshow(result_image)

The final result can be seen as:

My most favorite show to date and nothing can beat its flow of events and quirky comedy. Amazing performance coupled with a great story and accurate depiction of the tech scene in Silicon Valley has been Ended :worried:  peace!

Conclusion:

We can use this simple technique to achieve the same feature similar to youtube (low on cpu!) and can be tried even on video. In the next post, I will be moving towards the deep learning model and if you have any idea or would like to suggest some improvement, please do comment below.