Python with Statement and Context Managers

The with statement guarantees that resources such as files, network connections, and locks are set up and cleaned up correctly — even when an exception interrupts the block. The object that controls this setup and teardown is called a context manager.

This chapter explains how the with statement works, when to use it, how to write your own context managers using __enter__ and __exit__, and how to create lightweight ones with contextlib.contextmanager.

Why with Exists

Before the with statement, resource management meant writing try/finally blocks by hand:

f = open("data.txt", "r", encoding="utf-8")
try:
    content = f.read()
finally:
    f.close()   # must always close, even if read() raises

This works, but it is verbose, easy to forget, and adds boilerplate around every resource. The with statement condenses this into a single, readable block and handles the cleanup automatically:

with open("data.txt", "r", encoding="utf-8") as f:
    content = f.read()
# f is closed here, no matter what happened inside the block

The as f clause binds the context manager's value to the name f. Some context managers do not produce a useful value, in which case you can omit as:

with some_lock:
    shared_data.append(item)

How the with Statement Works

When Python executes a with statement, it follows this sequence:

1. Evaluate the expression after with — this produces the context manager object.
2. Call the context manager's __enter__() method. The return value of __enter__() is bound to the as variable (if present).
3. Run the body of the with block.
4. Call the context manager's __exit__(exc_type, exc_val, exc_tb) method.
   • If the block completed normally, all three arguments are None.
   • If an exception was raised, the three arguments describe it.
   • If __exit__ returns a truthy value, the exception is suppressed and execution continues after the with block. If it returns a falsy value (or None), the exception propagates.

This protocol is called the context manager protocol.

Opening Files with with

The most common use of with is file handling. Python's built-in file objects implement the context manager protocol, so they close automatically when the block ends:

with open("report.txt", "w", encoding="utf-8") as f:
    f.write("Sales: 1 000\n")
    f.write("Returns: 23\n")

print(f.closed)   # True — file was closed on exit

If an exception occurs inside the block, the file is still closed:

try:
    with open("data.txt", "r", encoding="utf-8") as f:
        raise RuntimeError("something went wrong")
except RuntimeError:
    pass

print(f.closed)   # True — closed despite the exception

Without with, forgetting f.close() after an error leaves the file descriptor open until the garbage collector runs — or until the process exits — which can cause data loss or "too many open files" errors in long-running programs.

Opening Multiple Resources at Once

You can open several resources in a single with statement by separating them with commas (Python 3.1+):

with open("input.txt", "r", encoding="utf-8") as src, \
     open("output.txt", "w", encoding="utf-8") as dst:
    for line in src:
        dst.write(line.upper())

This is exactly equivalent to nesting two with statements, but keeps the indentation level flat.

Writing a Context Manager with __enter__ and __exit__

Any class that defines __enter__ and __exit__ can be used with the with statement. Here is a minimal example — a timer that measures how long the with block runs:

import time

class Timer:
    def __enter__(self):
        self._start = time.perf_counter()
        return self                        # bound to the 'as' variable

    def __exit__(self, exc_type, exc_val, exc_tb):
        elapsed = time.perf_counter() - self._start
        print(f"Elapsed: {elapsed:.4f}s")
        return False                       # do not suppress exceptions

with Timer() as t:
    total = sum(range(1_000_000))

# Elapsed: 0.0xxx s
print(total)  # 499999500000

Key points:

• __enter__ runs before the block. It returns the value bound to as t. Returning self lets the caller access t.elapsed and other attributes if needed.
• __exit__ runs after the block, even on exception. Returning False (or None) lets any exception propagate normally.

Suppressing Exceptions in __exit__

If __exit__ returns True, the exception is swallowed and execution continues after the with block. This is intentional in specific contexts — for example, a context manager that catches and logs errors without crashing the program:

class Ignore:
    """Silently ignore any exception raised inside the with block."""

    def __enter__(self):
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type is not None:
            print(f"Suppressed: {exc_type.__name__}: {exc_val}")
        return True   # suppress the exception

with Ignore():
    x = 1 / 0        # ZeroDivisionError is caught and ignored

print("execution continues here")
# Suppressed: ZeroDivisionError: division by zero
# execution continues here

Use exception suppression carefully — silently swallowing errors can hide bugs. The standard library's contextlib.suppress is the idiomatic way to do this (see below).

A Database Connection Context Manager

A more realistic example — managing a database-style connection that commits on success and rolls back on error:

class ManagedTransaction:
    def __init__(self, connection):
        self.conn = connection

    def __enter__(self):
        self.conn.begin()
        return self.conn

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type is None:
            self.conn.commit()
        else:
            self.conn.rollback()
        return False   # always let exceptions propagate

The pattern — commit on success, rollback on failure — appears throughout real database libraries (SQLite, SQLAlchemy, psycopg2 all implement it).

contextlib.contextmanager: Generator-Based Context Managers

Writing a full class with __enter__ and __exit__ is the right approach for complex or stateful context managers. For simpler cases, the contextlib.contextmanager decorator lets you express the same logic as a generator function:

from contextlib import contextmanager

@contextmanager
def managed_open(path, mode="r", encoding="utf-8"):
    print(f"Opening {path}")
    f = open(path, mode, encoding=encoding)
    try:
        yield f          # everything up to yield is __enter__
    finally:
        f.close()        # everything after yield is __exit__
        print(f"Closed {path}")

with managed_open("notes.txt", "w") as f:
    f.write("hello\n")
# Opening notes.txt
# Closed notes.txt

The generator protocol maps directly onto the context manager protocol:

• Code before yield → __enter__ (setup).
• The yield expression → the value bound to the as variable.
• Code after yield (usually in a finally) → __exit__ (teardown).

The try/finally around yield is important: without it, an exception inside the with block would cause teardown code never to run.

contextmanager Example: Temporary Working Directory

import os
from contextlib import contextmanager

@contextmanager
def working_directory(path):
    original = os.getcwd()
    os.chdir(path)
    try:
        yield
    finally:
        os.chdir(original)

with working_directory("/tmp"):
    print(os.getcwd())   # /tmp (or system temp dir)

print(os.getcwd())       # restored to original directory

This pattern is also available in the standard library as tempfile.TemporaryDirectory.

contextlib Utilities

The contextlib module ships several ready-made context managers worth knowing:

contextlib.suppress

Suppress specific exceptions without any boilerplate:

from contextlib import suppress

with suppress(FileNotFoundError):
    os.remove("temp.txt")   # no error even if file does not exist

Equivalent to a try/except that does nothing on the caught exception.

contextlib.nullcontext

A no-op context manager useful when you conditionally want a context manager or not:

from contextlib import nullcontext

def process(data, lock=None):
    ctx = lock if lock is not None else nullcontext()
    with ctx:
        return sorted(data)

Without nullcontext, you would need an if lock: branch every time.

contextlib.ExitStack

ExitStack lets you manage a dynamic number of context managers — useful when the number of resources is not known until runtime:

from contextlib import ExitStack

files = ["a.txt", "b.txt", "c.txt"]

with ExitStack() as stack:
    handles = [
        stack.enter_context(open(f, "w", encoding="utf-8"))
        for f in files
    ]
    for i, fh in enumerate(handles):
        fh.write(f"file {i}\n")
# All three files are closed here

ExitStack is also the right tool when you need to conditionally add a context manager, or when you want to defer cleanup to a later point.

When to Use with vs. Try/Finally

Use with whenever:

• A resource must be released after use (files, sockets, locks, database cursors).
• You want to guarantee cleanup even on exceptions.
• The cleanup logic is always the same regardless of success or failure.

Use a plain try/finally only when:

• You need different cleanup actions depending on the exception type — though __exit__ can do this too.
• You are writing Python 2-compatible code (rare today).

In practice, if the object supports the context manager protocol, always prefer with.

Quick Reference

Related Chapters

• Python File Handling — using with open(...) for every file operation
• Python Try Except — exception handling and try/finally
• Python Decorators — @contextmanager uses the same decorator pattern
• Python Generators — how the generator protocol powers @contextmanager
• Python Classes and Objects — writing __enter__ and __exit__ methods

Practice