Visualize Missing Values with Missingno

Visualize Missing Values with Missingno

Explore the missing values in your dataset.

Data is the new fuel. However, the raw data is cheap. We need to process it well to take the most value out of it. Complex, well-structured models are as good as the data we feed to it. Thus, data needs to be cleaned and processed thoroughly in order to build robust and accurate models.

One of the issues that we are likely to encounter in raw data is missing values. Consider a case where we have features (columns in a dataframe) on some observations (rows in a dataframe). If we do not have the value in a particular row-column pair, then we have a missing value. We may have only a few missing values or half of an entire column may be missing. In some cases, we can just ignore or drop the rows or columns with missing values. On the other, there might be some cases in which we cannot afford to drop even a single missing value. In any case, handling missing values process starts with exploring them in the dataset.

Pandas provides functions to check the number of missing values in the dataset. Missingno library takes it one step further and provides the distribution of missing values in the dataset by informative visualizations. Using the plots of missingno , we are able to see where the missing values are located in each column and if there is a correlation between missing values of different columns. Before handling missing values, it is very important to explore them in the dataset. Thus, I consider missingno as a highly valuable asset in data cleaning and preprocessing steps.

In this post, we will explore the functionalities of missingno plot by going through some examples.

Let’s first try to explore a dataset about the movies on streaming platforms. The dataset is available here on kaggle.

import numpy as np
import pandas as pddf = pd.read_csv("/content/MoviesOnStreamingPlatforms.csv")
print(df.shape)
df.head()