P

Ranges and Generators

The Python range() function allows the user to create a sequence of values by providing a starting value, an ending value, and if needed, a step value. It is very similar to the slicing syntax in Appendix L. By default, if we give range() a single number, this function will create a sequence of values starting from 0.

# create a range of 5
r = range(5)

However, the range() function doesn’t just return a list of numbers. In Python 3, it actually returns a generator.

print(r)

range(0, 5)

print(type(r))

<class 'range'>

If we wanted an actual list of the range, we can convert the generator to a list.

lr = list(range(5))
print(lr)

[0, 1, 2, 3, 4]

Before you decide to convert a generator, you should think carefully about what you plan to use it for. If you plan to create a generator that will look over a set of data (Appendix M), then there is no need to convert the generator.

for i in lr:
  print(i)

0
1
2
3
4

Generators create the next value in the sequence on the fly. As a consequence, the entire contents of the generator do not need to be loaded into memory before using it. Since generators know only the current position and how to calculate the next item in the sequence, you cannot use generators a second time.

The following example comes from the built-in itertools library in Python. It creates a Cartesian product of values provided to the function.

Click here to view code image

import itertools
prod = itertools.product([1, 2, 3], ['a', 'b', 'c'])

for i in prod:
  print(i)

(1, 'a')
(1, 'b')
(1, 'c')
(2, 'a')
(2, 'b')
(2, 'c')
(3, 'a')
(3, 'b')
(3, 'c')

If you need to reuse the Cartesian product again, then you would have to either re-create the generator object or convert the generator into something more static (e.g., a list).

Click here to view code image

# this will not work because we already used this generator
for i in prod:
  print(i)

# create a new generator
prod = itertools.product([1, 2, 3], ['a', 'b', 'c'])
for i in prod:
  print(i)

(1, 'a')
(1, 'b')
(1, 'c')
(2, 'a')
(2, 'b')
(2, 'c')
(3, 'a')
(3, 'b')
(3, 'c')

If all you are doing is creating something to iterate over once, it will save you a lot of computer memory if you do not convert it into a list object, since Python will just create the object as it goes, instead of trying to store the entire thing at once.