List Comprehensions in Python — Explained

List Comprehensions in Python — Explained

When to use and not to use list comprehensions.

Listis a built-in data structure in Python and a collection of data points in square brackets. Other built-in data structures in Python are set , tuple , and dictionary .

list_a = [4, 'data', 3.2, True]list_a
[4, 'data', 3.2, True]

In this post, we will cover list comprehensions in python and their advantages. Although list comprehensions are highly practical, there are cases in which list comprehension is not the optimal choice. We will also go through situations when not to use list comprehensions.

List comprehension is basically creating lists based on existing lists. Following is a list comprehension that creates a list based on the lenghts of words in another list.

words = ['data', 'science', 'machine', 'learning']word_length = [len(i) for i in words]word_length
[4, 7, 7, 8]

The basic syntax for the list comprehension is:

In the previous example, expression is len(i), item is the elements in “words” list represented by “i”. The iterable is, of course, the “words” list. We did not have a conditional statement but let’s do another one with a condition. For instance, the following list comprehension creates a list with the words with a length greater than 5.

words = ['data', 'science', 'machine', 'learning']long_words = [i for i in words if len(i) > 5]long_words
['science', 'machine', 'learning']

Expression can be any expression that returns a value. Iterable can be any object that can iteratively return its elements such as a list, set, generator.

Conditionals are critical because they allow to filter out values or select only what we need.

For instance, the following piece of code creates a list consists of the squares of even numbers in range(20).

even_squares = [i*i for i in range(20) if i%2 == 0]even_squares
[0, 4, 16, 36, 64, 100, 144, 196, 256, 324]

We can create a list with the maximum numbers of rows of a matrix.

#Create a matrix
import numpy as np
A = np.random.randint(10, size=(4,4))
A
array([[1, 7, 4, 4],        
       [5, 0, 0, 6],        
       [7, 5, 8, 4],        
       [1, 3, 2, 2]])#select the max of rows
max_element = [max(i) for i in A]
max_element
[7, 6, 8, 3]

We can also put a condition on the expression. The following piece of code iterates over “words” list. It gets the words if the length is greater than 7. It writes “short word” instead of the elements whose length is not greater than 7.

words = ['data', 'science', 'artificial', 'intelligence', 'machine', 'learning']long_words = [i if len(i) > 7 else 'short word' for i in words]print(long_words)
['short word', 'short word', 'artificial', 'intelligence', 'short word', 'learning']

List comprehension vs for loop vs map

What we do with list comprehensions can also be done with for loops or map function. Let’s do the first example using both for loop and map function:

#for loop
word_length = []
for word in words:
   word_length.append(len(word))word_length
[4, 7, 7, 8]#map function
word_length = list(map(lambda x: len(x), words))word_length
[4, 7, 7, 8]

The advantage to use list comprehensions:

• They are relatively faster than for loops.
• They are considered to be more pythonic than for loop and map function.
• The syntax of list comprehension is easier to read and understand.

Let’s do a comparison by creating a list with the squares of first 50000 integers:

As we can see, the list comprehension is the fastest.

Note: Every list comprehension can be written using a for loop but not every for loop can be represented with a list comprehension.

When not to use list comprehension

List comprehension loads the entire output list into memory. This is acceptable or even desirable for small or medium-sized lists because it makes the operation faster. However, when we are working with large lists (e.g. 1 billion elements), list comprehension should be avoided. It may cause your computer to crash due to the extreme amount of memory requirement.

A better alternative for such large lists is using a generator which does not actually create a large data structure in memory. A generator creates items when they are used. After the items are used, generator throws them away. Using a generator, we can ask for the next item in an iterable until we reach the end and store a single value at a time.

The following generator sums the squares of first 10 million integers.

sum(i*i for i in range(10000000))333333283333335000000

Map function does not also cause a memory problem.

sum(map(lambda i: i*i, range(10000000)))333333283333335000000

There is no free lunch! Generators or map functions do not cause a memory issue but they’re relatively slower than list comprehensions. Similarly, the speed of list comprehensions comes from the excessive memory usage. You can decide on which one to use depending on your application.