Semantic Segmentation of 150 classes of objects With 5 Lines of Code

Semantic Segmentation of 150 classes of objects With 5 Lines of Code

Semantic segmentation of 150 classes of objects with PixelLib

It is now possible to perform segmentation on 150 classes of objects using ade20k model with PixelLib. Ade20k model is a deeplabv3+ model trained on ade20k dataset, a dataset with 150 classes of objects. Thanks to tensorflow deeplab’s model zoo, I extracted ade20k model from its tensorflow model checkpoint.

Install the latest version tensorflow(tensorflow 2.0) with:

Install Pixellib:

• pip3 install pixellib — upgrade

Implementation of Semantic Segmentation with PixelLib:

The code to implement semantic segmentation with deeplabv3+ model is trained on ade20k dataset.

We shall observe each line of code:

import pixellib
from pixellib.semantic import semantic_segmentation segment_image = semantic_segmentation()

The class for performing semantic segmentation is imported from pixelLib and we created an instance of the class.

segment_image.load_ade20k_model(“deeplabv3_xception65_ade20k.h5”)

In the code above we loaded the xception model trained on ade20k for segmenting objects. The model can be downloaded from here .

segment_image.segmentAsAde20k(“path_to_image”, output_image_name = “path_to_output_image)

We loaded the function to perform segmentation on an image. The function takes two parameters…

• path_to_image:- this is the path to the image to be segmented.
• output_image_name:- this is the path to save the segmented image. It will be saved in your current working directory.

Sample.jpg

Note: It is possible to perform semantic segmentation of both indoor and outdoor scenes with PixelLib using Ade20k model.

Ouput image

Semantic segmentation of an outdoor scene

The saved image after segmentation, the objects in the image are segmented. You can apply segmentation overlay on the image if you want to.

segment_image.segmentAsAde20k("sample.jpg", output_image_name = "image_new.jpg", overlay = True)

We added the extra parameter overlay and set it to true and we obtained an image with a segmentation overlay on the objects.

Sample2.jpg

segment_image.segmentAsAde20k(“sample2.jpg”, output_image_name = “image_new2.jpg")

Output image

Semantic segmentation of an indoor scene

Specialised uses of PixelLib may require you to return the array of the segmentation’s output:

Obtain the array of the segmentation’s output by using this code,

segmap, output = segment_image.segmentAsAde20k()

You can test the code for obtaining arrays and print out the shape of the output by modifying the semantic segmentation code below.

• Obtain both the segmap and the segmentation overlay’s arrays by using this code,

segmap, seg_overlay = segment_image.segmentAsAde20k(overlay = True)

VIDEO SEGMENTATION WITH ADE20K MODEL

We shall explain each line of code below.

import pixellibfrom pixellib.semantic import semantic_segmentationsegment_video = semantic_segmentation()

We imported in the class for performing semantic segmentation and created an instance of the class.

segment_video.load_ade20k_model("deeplabv3_xception65_ade20k.h5")

We loaded the xception model trained on ade20k dataset to perform semantic segmentation and it can be downloaded from here .

segment_video.process_video_ade20k("video_path",  overlay = True, frames_per_second= 15, output_video_name="path_to_output_video")

We called the function to perform segmentation on the video file.

It takes the following parameters:-

• video_path :this is the path to the video file we want to perform segmentation on.
• frames_per_second: this is the parameter used to set the number of frames per second for the saved video file. In this case it is set to 15 i.e the saved video file will have 15 frames per second.
• output_video_name: this is the name of the saved segmented video . The output video will be saved in your current working directory.

sample_video

Output Video

This is the saved segmented video using ade20k model.

Semantic Segmentation of Live Camera.

We can use the same model to perform semantic segmentation on camera. This can be done by including few modifications to the code that is used to process a video file.

import cv2capture = cv2.VideoCapture(0)

We imported cv2 and included the code to capture camera’s frames.

segment_video.process_camera_ade20k(capture,  overlay = True, frames_per_second= 15, output_video_name="output_video.mp4", show_frames= True,frame_name= "frame", check_fps = True)

In the code for performing segmentation, we replaced the video’s filepath to capture i.e we are processing a stream of frames captured by the camera instead of a video file.We added extra parameters for the purpose of showing the camera frames:

• show_frames: this parameter handles showing of segmented camera’s frames and press q to exist the display of frames.
• frame_name: this is the name given to the shown camera’s frame.

check_fps: You may want to check the number of frames processed per second, just set the parameter check_fps to true .It will print out the number of frames per seconds. In this case it is 30 frames per second .

Awesome! 30 frames per second is great for Real Time Segmentation of camera’s feed.

A demo showing the output of pixelLib’s semantic segmentation on camera’s feeds using ade20k model.

Good work! It successfully segmented me.

Visit the official github repository of PixelLib.

Visit the official documentation of PixelLib

Reach to me via:

Email: olafenwaayoola@gmail.com

Twitter: @AyoolaOlafenwa

Facebook: Ayoola Olafenwa

Linkedin: Ayoola Olafenwa

Read these other articles about PixelLib:

Image Segmentation With 5 Lines of code

Video Segmentation With 5 Lines of code