Python Iterators

An iterator is one of Python's most fundamental abstractions. Whenever you write a for loop, call zip(), or use a list comprehension, Python silently relies on the iterator protocol under the hood. This chapter explains what iterators are, how to build your own, how to use the rich set of built-in iterators, and when iterators are the right tool for the job.

What Is an Iterator?

Python distinguishes between two related concepts:

• An iterable is any object you can loop over — a list, tuple, str, dict, set, or any object whose class defines __iter__. It can produce an iterator, but it does not track position itself.
• An iterator is an object that tracks traversal state. It implements two methods that together form the iterator protocol:
  • __iter__() — returns the iterator object itself. This makes iterators work inside for loops and other iteration contexts.
  • __next__() — returns the next value each time it is called. When no values remain, it raises StopIteration.

The key difference: you can loop over a list as many times as you like because each for loop asks for a fresh iterator. An iterator is one-way and one-time — once exhausted, calling next() on it always raises StopIteration.

graph LR
  A[Iterator Object] --> B[__iter__]
  B --> C[Returns self]
  A --> D[__next__]
  D --> E[Next Value]
  D --> F{No values left?}
  F -->|Yes| G[Raises StopIteration]
  F -->|No| E

How a for Loop Uses Iterators

The for loop is just syntactic sugar for the iterator protocol. Internally, Python translates:

for item in some_iterable:
    print(item)

into roughly this:

_it = iter(some_iterable)   # call __iter__()
while True:
    try:
        item = next(_it)    # call __next__()
    except StopIteration:
        break
    print(item)

Understanding this translation makes it clear why any object that implements __iter__ and __next__ works seamlessly in a for loop, with zip(), enumerate(), and any other context that expects an iterable.

Building a Custom Iterator

To create a custom iterator, define a class that implements both __iter__ and __next__. Here is a Countdown iterator that counts down from a given number to 1:

class Countdown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self        # the iterator is its own iterable

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value

for n in Countdown(5):
    print(n)
# Output:
# 5
# 4
# 3
# 2
# 1

Notice that __iter__ returns self. This is what allows the same object to be placed directly in a for loop — the loop calls iter() on it, which calls __iter__(), which returns the iterator itself.

Adding a Step Parameter

You can add any logic you like inside __next__. Here is a StepRange iterator that mimics range() but accepts a step value:

class StepRange:
    def __init__(self, start, stop, step=1):
        self.current = start
        self.stop = stop
        self.step = step

    def __iter__(self):
        return self

    def __next__(self):
        if self.current >= self.stop:
            raise StopIteration
        value = self.current
        self.current += self.step
        return value

print(list(StepRange(0, 10, 3)))
# Output: [0, 3, 6, 9]

Calling list() on any iterator exhausts it and collects every value into a list — a useful pattern when you need all results at once.

The iter() and next() Built-ins

The built-in functions iter() and next() are the standard way to work with the iterator protocol directly.

• iter(obj) — calls obj.__iter__() and returns the resulting iterator.
• next(it) — calls it.__next__() and returns the next value.
• next(it, default) — returns default instead of raising StopIteration when the iterator is exhausted. This is the safest way to peek at the next element without a try/except block.

words = ["hello", "world"]
it = iter(words)

print(next(it))           # hello
print(next(it))           # world
print(next(it, "done"))   # done  (exhausted; returns default)

The two-argument form of next() is particularly useful in streaming or parsing scenarios where you want to handle the end of input gracefully.

Iterators Are One-Time Use

This is the most common gotcha with iterators: once exhausted, an iterator cannot be rewound.

it = iter([1, 2, 3])

for x in it:
    print(x)        # prints 1, 2, 3

for x in it:
    print(x)        # prints nothing — iterator is exhausted

If you need to iterate multiple times, keep the original iterable (e.g., the list) and call iter() again, or use a list comprehension to materialise all values first.

Built-in Functions That Return Iterators

Python's standard library is built on iterators. These functions all return iterators rather than lists, so they are memory-efficient even over very large sequences:

range()

range(start, stop, step) returns an iterator of integers. It does not store the integers in memory — it computes each one on demand.

for i in range(1, 6):
    print(i)
# Output: 1  2  3  4  5

zip()

zip() takes multiple iterables and returns an iterator of tuples, pairing up elements position-by-position. Iteration stops at the shortest input.

names = ["Alice", "Bob", "Carol"]
scores = [95, 88, 72]

for name, score in zip(names, scores):
    print(f"{name}: {score}")
# Output:
# Alice: 95
# Bob: 88
# Carol: 72

enumerate()

enumerate() wraps any iterable and returns (index, value) pairs. Use it to avoid maintaining a manual counter variable.

fruits = ["apple", "banana", "cherry"]

for i, fruit in enumerate(fruits, start=1):
    print(f"{i}. {fruit}")
# Output:
# 1. apple
# 2. banana
# 3. cherry

map() and filter()

Both functions return iterators (in Python 3). map(fn, iterable) applies a function to every element; filter(fn, iterable) keeps only elements for which the function returns True.

numbers = [1, 2, 3, 4, 5]

doubled = list(map(lambda x: x * 2, numbers))
print(doubled)          # [2, 4, 6, 8, 10]

evens = list(filter(lambda x: x % 2 == 0, numbers))
print(evens)            # [2, 4]

Checking Whether an Object Is an Iterator

Use isinstance() with the abstract base classes from the collections.abc module to test for iterability and iterator status:

from collections.abc import Iterable, Iterator

my_list = [1, 2, 3]
my_iter = iter(my_list)

print(isinstance(my_list, Iterable))   # True  — list is iterable
print(isinstance(my_list, Iterator))   # False — list is NOT an iterator
print(isinstance(my_iter, Iterator))   # True  — list_iterator is an iterator
print(isinstance(my_iter, Iterable))   # True  — all iterators are also iterables

Every iterator is also an iterable (because __iter__ returns self), but not every iterable is an iterator.

When to Use Iterators vs. Lists

For large datasets — reading millions of rows from a file, processing streaming data — an iterator avoids loading everything into memory at once. For small, finite collections where you access elements repeatedly, a list is simpler.

Iterators vs. Generators

A generator is a convenient shorthand for writing an iterator. Instead of a class with __iter__ and __next__, you write a function that uses yield. Python converts it into an iterator automatically.

# Iterator class
class Countdown:
    def __init__(self, start):
        self.current = start
    def __iter__(self):
        return self
    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value

# Equivalent generator function
def countdown(start):
    while start > 0:
        yield start
        start -= 1

print(list(countdown(5)))   # [5, 4, 3, 2, 1]

Use a class-based iterator when you need additional methods or mutable state beyond what a simple generator offers. Use a generator for most other cases — it is more concise and equally powerful.

See the Python Generators chapter for a full treatment of yield, generator expressions, and send().

Practice