Bye-bye MobileNet. Hello EfficientNet!

Opinion

Bye-bye MobileNet. Hello EfficientNet!

MobileNet was the go-to model for an app and edge deployment. Now it is dethroned by the family of EfficientNet Lite models.

How to run complex deep learning models on mobiles and edge devices with a limitation of processing power and memory with good speed? Create a model with a backbone of MobileNetV2, convert it to Tensorflow Lite, and you are done. Now it is challenged by EfficientNet Lite. Read on to learn about the need for EfficientNet-Lite from EfficientNet, how to create EfficientNet Lite models and we will also compare these models to see who reigns supreme. If you want to learn about the architecture of EfficientNet in-depth, you can read the article below.

Before I start, Yes I have shamelessly stolen the title from Rhea Moutafis hit article Bye-bye Python. Hello Julia!

Drawbacks of EfficientNet models in Tensorflow Lite → Need of EfficientNet Lite

Recently, I had written an article comparing EfficientNet against other pre-trained models like MobileNetV2, Inception, and Xception and I thought of converting those saved models to their Tensorflow Lite counterparts to see how their inference time stacks against each other.

It was all fine till comparing the model sizes but after that, I was in for a shock.

To calculate the inference time I loaded and processed an image and made predictions on it for a hundred times and took its average.

The inference time for MobileNet dropped for the Tensorflow Lite model as expected but it increased for the EfficientNet models!!! Tensorflow Lite should make the models smaller and decrease inference time! So why is this happening and how to solve this? This is cleared up in EfficientNet Lite which has the following changes according to this article :

• Removed squeeze-and-excitation networks since they are not well supported
• Replaced all swish activations with RELU6, which significantly improved the quality of post-training quantization
• Fixed the stem and head while scaling models up to reduce the size and computations of scaled models

MobileNet VS EfficientNet Lite in Tensorflow Lite

These models will be created with model maker which as stated in its tutorial

Model Maker library simplifies the process of adapting and converting a TensorFlow neural-network model to particular input data when deploying this model for on-device ML applications.

It sure does this as and creating and it supports MobileNetV2, ResNet50, and the first five models of EfficientNet Lite family. You can also you different models from the Tensorflow Hub using ImageModelSpec or create or own custom model and export ModelSpec to Tensorflow Hub and then use ImageModelSpec .

In this article, I will only be combining MobileNetV2 with EfficientNet Lite 0 to 4. The data used would be the flower dataset with labels of daisy, dandelion, roses, sunflowers, and tulips which will be divided into 80, 10, 10% for training, validation, and testing respectively.

We will need Tensorflow examples for this which can be pip installed using:

!pip install -q git+https://github.com/tensorflow/examples.git#egg=tensorflow-examples[model_maker]

Then we make our required imports which are the model specs, image classifier, data loader for image classifier, TensorFlow, NumPy, and Matplotlib.

from tensorflow_examples.lite.model_maker.core.data_util.image_dataloader import ImageClassifierDataLoader
from tensorflow_examples.lite.model_maker.core.task import image_classifier
from tensorflow_examples.lite.model_maker.core.task.model_spec import (mobilenet_v2_spec,
        efficientnet_lite0_spec,
        efficientnet_lite1_spec,
        efficientnet_lite2_spec,
        efficientnet_lite3_spec,
        efficientnet_lite4_spec)
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

Then we load the data, split it into the required categories, and use the ImageClassifierDataLoader class to make it ready for the image classifier. The from_folder method can be used and it assumes that images of different classes are present in different subfolders in the main folder with the name of the subfolder being the class name. Make sure the images have a PNG or JPG extension as only they are supported.

image_path = tf.keras.utils.get_file('flower_photos',
'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz', untar=True)data = ImageClassifierDataLoader.from_folder(image_path)
train_data, rest_data = data.split(0.8)
validation_data, test_data = rest_data.split(0.5)

Let's take a look at our data

Creating models with model-maker is one-liner now.

model = image_classifier.create(train_data, model_spec=model_spec, epochs=epochs, validation_data=validation_data)

Specify whichever model spec you want like for MobileNetV2 it is mobilenet_v2_spec or for EfficientNet Lite-2 it is efficientnet_lite2_spec as stated in the imports. EfficientNet Lite-0 is the default one if no one is specified. I trained each for 15 epochs and here are the results.

Surprisingly, EfficientNet Lite-4 performed badly on both the testing and training sets but this might just mean that it needs more training epochs. The worst performance on the validation set was of MobileNet and the other EfficientNet models were close to each other with EfficientNet Lite-2 and EfficientNet Lite-3 sharing the spoils with the highest accuracy.

To convert these models and save them as Tensorflow Lite files write

model.export(export_dir='.')

This saves a label.txt and a model.tflite file. Using these models is done like the normal tflite models by creating an interpreter.

# Read TensorFlow Lite model from TensorFlow Lite file.
with tf.io.gfile.GFile('model.tflite', 'rb') as f:
  model_content = f.read()

# Read label names from label file.
with tf.io.gfile.GFile('labels.txt', 'r') as f:
  label_names = f.read().split('\n')

# Initialze TensorFlow Lite inpterpreter.
interpreter = tf.lite.Interpreter(model_content=model_content)
interpreter.allocate_tensors()
input_index = interpreter.get_input_details()[0]['index']
output = interpreter.tensor(interpreter.get_output_details()[0]["index"])

# Run predictions on each test image data and calculate accuracy.
accurate_count = 0
for i, (image, label) in enumerate(test_data.dataset):
    # Pre-processing should remain the same. Currently, just normalize each pixel value and resize image according to the model's specification.
    image, _ = model.preprocess(image, label)
    # Add batch dimension and convert to float32 to match with the model's input
    # data format.
    image = tf.expand_dims(image, 0).numpy()

    # Run inference.
    interpreter.set_tensor(input_index, image)
    interpreter.invoke()

    # Post-processing: remove batch dimension and find the label with highest
    # probability.
    predict_label = np.argmax(output()[0])
    # Get label name with label index.
    predict_label_name = label_names[predict_label]

    accurate_count += (predict_label == label.numpy())

accuracy = accurate_count * 1.0 / test_data.size
print('TensorFlow Lite model accuracy = %.3f' % accuracy)

The size of the models, the testing accuracies, and the inference time which was again a mean of 100 times were noted down.