Python itertools Module

Python's itertools module is a standard-library toolkit of fast, memory-efficient building blocks for working with iterators. Every function in itertools returns an iterator — it produces values on demand rather than building a list in memory — making the module ideal for large datasets, infinite sequences, and composable data pipelines.

This chapter covers all three categories of itertools functions: infinite iterators (count, cycle, repeat), combinatoric iterators (product, permutations, combinations, combinations_with_replacement), and terminating iterators (chain, islice, groupby, compress, filterfalse, takewhile, dropwhile, starmap, zip_longest, accumulate, pairwise).

No installation is needed — itertools ships with every Python 3 installation:

import itertools

Why itertools?

Consider reading the first 10 multiples of a number. Without itertools you need a list or a manual counter. With itertools.count and itertools.islice the intent is immediately clear and memory usage stays constant:

import itertools

multiples = itertools.islice(itertools.count(0, 7), 10)
print(list(multiples))
# [0, 7, 14, 21, 28, 35, 42, 49, 56, 63]

The itertools philosophy: build a small, correct piece, then compose it with others. Chaining two itertools functions is faster and less error-prone than writing the equivalent loop by hand.

Infinite Iterators

These iterators produce values indefinitely. Always pair them with islice, a for … break, or another terminating mechanism to avoid an infinite loop.

count(start=0, step=1)

count produces an evenly-spaced sequence of numbers. It is essentially range with no upper bound and support for floats and negative steps.

import itertools

# Integer counter
for n in itertools.islice(itertools.count(10), 5):
    print(n, end=' ')
# 10 11 12 13 14

print()

# Float step
for n in itertools.islice(itertools.count(0.0, 0.5), 5):
    print(n, end=' ')
# 0.0 0.5 1.0 1.5 2.0

print()

# Countdown
for n in itertools.islice(itertools.count(100, -10), 5):
    print(n, end=' ')
# 100 90 80 70 60

count is useful when you need to number items from an iterable without knowing how many there are upfront — the enumerate idiom but with a custom start and step.

cycle(iterable)

cycle repeats the elements of any iterable indefinitely.

import itertools

colours = itertools.cycle(['red', 'green', 'blue'])
for i, colour in enumerate(colours):
    if i == 7:
        break
    print(colour, end=' ')
# red green blue red green blue red

Practical use — assigning items to teams in round-robin fashion:

import itertools

teams = itertools.cycle(['Alpha', 'Beta', 'Gamma'])
players = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']
assignments = {player: team for player, team in zip(players, teams)}
print(assignments)
# {'Alice': 'Alpha', 'Bob': 'Beta', 'Carol': 'Gamma', 'Dave': 'Alpha', 'Eve': 'Beta'}

repeat(object, times=None)

repeat yields the same object times times (or forever if times is omitted).

import itertools

# Finite repeat
print(list(itertools.repeat('hello', 3)))
# ['hello', 'hello', 'hello']

# Used as a fixed argument supplier in map()
squares = list(map(pow, range(1, 6), itertools.repeat(2)))
print(squares)
# [1, 4, 9, 16, 25]

The map(pow, range(1, 6), repeat(2)) pattern is a common idiom for supplying a constant second argument to a two-argument function.

Combinatoric Iterators

These iterators produce all combinations, permutations, or cross-products of an input iterable. They are essential for brute-force searches, test case generation, and combinatorics problems.

product(*iterables, repeat=1)

product computes the Cartesian product — all ordered combinations where one element is drawn from each iterable. It is equivalent to nested for loops.

import itertools

suits = ['Hearts', 'Diamonds']
ranks = ['A', 'K', 'Q']
deck = list(itertools.product(suits, ranks))
print(deck)
# [('Hearts', 'A'), ('Hearts', 'K'), ('Hearts', 'Q'),
#  ('Diamonds', 'A'), ('Diamonds', 'K'), ('Diamonds', 'Q')]

Use repeat to compute the product of an iterable with itself multiple times:

import itertools

# All 2-digit binary numbers
binary_pairs = list(itertools.product([0, 1], repeat=2))
print(binary_pairs)
# [(0, 0), (0, 1), (1, 0), (1, 1)]

Gotcha: product materialises the input iterables into memory (to allow multiple passes), so do not pass enormous iterators as input.

permutations(iterable, r=None)

permutations produces all ordered arrangements of r elements taken from the input. When r is omitted, all elements are used.

import itertools

# All orderings of 3 letters
perms = list(itertools.permutations('ABC'))
print(perms)
# [('A', 'B', 'C'), ('A', 'C', 'B'), ('B', 'A', 'C'),
#  ('B', 'C', 'A'), ('C', 'A', 'B'), ('C', 'B', 'A')]
print(len(perms))   # 6  (3! = 6)

# 2-element permutations
perms2 = list(itertools.permutations('ABC', 2))
print(perms2)
# [('A', 'B'), ('A', 'C'), ('B', 'A'), ('B', 'C'), ('C', 'A'), ('C', 'B')]
print(len(perms2))  # 6  (3 * 2 = 6)

Order matters in permutations — ('A', 'B') and ('B', 'A') are distinct results.

combinations(iterable, r)

combinations produces all unordered selections of r elements. Unlike permutations, order does not matter — each subset appears only once.

import itertools

# All 2-element subsets of [1, 2, 3, 4]
combos = list(itertools.combinations([1, 2, 3, 4], 2))
print(combos)
# [(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)]
print(len(combos))  # 6  (C(4,2) = 6)

A common use case — checking all pairs of items for a property:

import itertools

words = ['bat', 'tab', 'cat', 'tac']
anagram_pairs = [
    (a, b) for a, b in itertools.combinations(words, 2)
    if sorted(a) == sorted(b)
]
print(anagram_pairs)
# [('bat', 'tab'), ('cat', 'tac')]

combinations_with_replacement(iterable, r)

Like combinations, but allows each element to appear more than once in a selection.

import itertools

# All 2-element combinations with repetition from [1, 2, 3]
combos = list(itertools.combinations_with_replacement([1, 2, 3], 2))
print(combos)
# [(1, 1), (1, 2), (1, 3), (2, 2), (2, 3), (3, 3)]

This is useful for generating all possible dice rolls, coin-flip sequences, or password character choices.

Combinatoric functions at a glance

Terminating Iterators

Terminating iterators process a finite input and stop when that input is exhausted.

chain(*iterables)

chain treats several iterables as a single continuous sequence without building a new list.

import itertools

a = [1, 2, 3]
b = (4, 5)
c = range(6, 9)
combined = list(itertools.chain(a, b, c))
print(combined)
# [1, 2, 3, 4, 5, 6, 7, 8]

chain.from_iterable accepts a single iterable of iterables — useful when you do not know the number of sequences ahead of time:

import itertools

nested = [[1, 2], [3, 4], [5, 6]]
flat = list(itertools.chain.from_iterable(nested))
print(flat)
# [1, 2, 3, 4, 5, 6]

This is a fast, memory-efficient alternative to [item for sublist in nested for item in sublist].

islice(iterable, stop) / islice(iterable, start, stop, step=1)

islice slices any iterator — including infinite ones — without materialising it. The arguments mirror Python's slice notation but accept only non-negative integers.

import itertools

# First 5 elements
print(list(itertools.islice(range(100), 5)))
# [0, 1, 2, 3, 4]

# Elements 10–14 (start inclusive, stop exclusive)
print(list(itertools.islice(range(100), 10, 15)))
# [10, 11, 12, 13, 14]

# Every other element from position 0 to 10
print(list(itertools.islice(range(20), 0, 10, 2)))
# [0, 2, 4, 6, 8]

islice does not support negative indices or negative steps (unlike regular list slicing).

groupby(iterable, key=None)

groupby groups consecutive elements that share the same key value. It returns (key, group_iterator) pairs.

import itertools

data = [
    ('fruit', 'apple'),
    ('fruit', 'banana'),
    ('veggie', 'carrot'),
    ('veggie', 'broccoli'),
    ('fruit', 'cherry'),
]

for category, group in itertools.groupby(data, key=lambda x: x[0]):
    items = [item[1] for item in group]
    print(f'{category}: {items}')
# fruit: ['apple', 'banana']
# veggie: ['carrot', 'broccoli']
# fruit: ['cherry']

Critical gotcha: groupby only groups consecutive equal elements. If your data is not pre-sorted by the key, similar items in different positions form separate groups (as shown above — 'cherry' starts a new 'fruit' group instead of joining the first). Always sort by the key before calling groupby:

import itertools

data = [
    ('fruit', 'apple'),
    ('veggie', 'carrot'),
    ('fruit', 'banana'),
    ('veggie', 'broccoli'),
    ('fruit', 'cherry'),
]

# Sort first, then group
sorted_data = sorted(data, key=lambda x: x[0])
for category, group in itertools.groupby(sorted_data, key=lambda x: x[0]):
    items = [item[1] for item in group]
    print(f'{category}: {items}')
# fruit: ['apple', 'banana', 'cherry']
# veggie: ['carrot', 'broccoli']

Also note that the group iterator becomes invalid once you advance to the next key — consume each group before calling next() on the outer iterator.

compress(data, selectors)

compress filters data by keeping only elements whose corresponding selector value is truthy.

import itertools

names = ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']
active = [True, False, True, True, False]

result = list(itertools.compress(names, active))
print(result)
# ['Alice', 'Carol', 'Dave']

compress is equivalent to [d for d, s in zip(data, selectors) if s] but is faster and avoids the intermediate list.

filterfalse(predicate, iterable)

filterfalse is the complement of the built-in filter — it yields elements for which the predicate returns False.

import itertools

numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# Keep only odd numbers (those that fail the even test)
odds = list(itertools.filterfalse(lambda x: x % 2 == 0, numbers))
print(odds)
# [1, 3, 5, 7, 9]

takewhile(predicate, iterable)

takewhile yields elements as long as the predicate is True, then stops immediately — even if later elements would satisfy the predicate.

import itertools

data = [2, 4, 6, 3, 8, 10]

# Stop as soon as an odd number appears
evens_from_start = list(itertools.takewhile(lambda x: x % 2 == 0, data))
print(evens_from_start)
# [2, 4, 6]

dropwhile(predicate, iterable)

dropwhile is the mirror of takewhile: it skips elements while the predicate is True, then yields all remaining elements (including those where the predicate would be True again).

import itertools

data = [2, 4, 6, 3, 8, 10]

# Drop leading even numbers, yield everything from the first odd onward
result = list(itertools.dropwhile(lambda x: x % 2 == 0, data))
print(result)
# [3, 8, 10]

takewhile and dropwhile are useful for processing log files or streams where you want to skip a header section or stop at a sentinel line.

starmap(function, iterable)

starmap applies a function to each element of an iterable, unpacking the element as positional arguments. It is the equivalent of map but for iterables of tuples.

import itertools

pairs = [(2, 3), (4, 2), (10, 3)]
results = list(itertools.starmap(pow, pairs))
print(results)
# [8, 16, 1000]

Compare with map(pow, [2, 4, 10], [3, 2, 3]) — starmap works when your arguments are already bundled as tuples.

zip_longest(*iterables, fillvalue=None)

The built-in zip stops at the shortest iterable. zip_longest pads shorter iterables with fillvalue so all iterables are fully consumed.

import itertools

a = [1, 2, 3]
b = ['a', 'b', 'c', 'd', 'e']

print(list(zip(a, b)))
# [(1, 'a'), (2, 'b'), (3, 'c')]  — b's 'd' and 'e' are lost

print(list(itertools.zip_longest(a, b, fillvalue=0)))
# [(1, 'a'), (2, 'b'), (3, 'c'), (0, 'd'), (0, 'e')]

accumulate(iterable, func=operator.add, *, initial=None)

accumulate computes running totals (or any other running aggregation). By default it sums, but you can pass any two-argument function.

import itertools
import operator

numbers = [1, 2, 3, 4, 5]

# Running sum (default)
print(list(itertools.accumulate(numbers)))
# [1, 3, 6, 10, 15]

# Running product
print(list(itertools.accumulate(numbers, operator.mul)))
# [1, 2, 6, 24, 120]

# Running maximum
data = [3, 1, 4, 1, 5, 9, 2, 6]
print(list(itertools.accumulate(data, max)))
# [3, 3, 4, 4, 5, 9, 9, 9]

The initial parameter (Python 3.8+) prepends a seed value before the first element:

import itertools

print(list(itertools.accumulate([1, 2, 3], initial=100)))
# [100, 101, 103, 106]

pairwise(iterable)

pairwise (Python 3.10+) yields consecutive overlapping pairs from the iterable.

import itertools

data = [1, 2, 3, 4, 5]
print(list(itertools.pairwise(data)))
# [(1, 2), (2, 3), (3, 4), (4, 5)]

This is useful for computing differences between consecutive values, or for sliding-window logic where the window size is exactly 2:

import itertools

prices = [10.0, 12.5, 11.0, 13.5, 15.0]
changes = [b - a for a, b in itertools.pairwise(prices)]
print(changes)
# [2.5, -1.5, 2.5, 1.5]

Before Python 3.10, the equivalent was zip(data, data[1:]) (works for sequences) or a manual tee-based approach (works for arbitrary iterators).

Composing itertools into Pipelines

The real power of itertools emerges when you combine functions. Because every function returns an iterator, you can chain them with zero intermediate lists.

Example: top-3 word frequencies in a text

import itertools
import operator

text = "the quick brown fox jumps over the lazy dog the fox"
words = text.split()

# Sort words so groupby can collect identical words together
sorted_words = sorted(words)

# Count each word using groupby
word_counts = (
    (key, sum(1 for _ in group))
    for key, group in itertools.groupby(sorted_words)
)

# Sort by count descending, take the top 3
top3 = list(itertools.islice(
    sorted(word_counts, key=operator.itemgetter(1), reverse=True),
    3
))
print(top3)
# [('the', 3), ('fox', 2), ('brown', 1)]

Example: batching an iterable into fixed-size chunks

import itertools

def batched(iterable, n):
    """Yield successive n-sized tuples from iterable."""
    it = iter(iterable)
    while chunk := tuple(itertools.islice(it, n)):
        yield chunk

data = range(10)
for batch in batched(data, 3):
    print(batch)
# (0, 1, 2)
# (3, 4, 5)
# (6, 7, 8)
# (9,)

Python 3.12 ships itertools.batched built-in, so you can replace the helper above with itertools.batched(data, 3) on modern Python.

Quick Reference

For the lazy-evaluation concepts behind itertools, see Python Generators and Python Iterators. For functional-style helpers that complement itertools, see Python Lambda Functions and the Python collections Module.

Practice