Python File Handling: A Comprehensive Guide

File handling lets your programs store data permanently and retrieve it later. This chapter explains every common file operation in Python — opening files in different modes, reading and writing data, navigating inside a file, specifying character encoding, and handling the errors that arise in real-world code.

The open() Function

Every file operation in Python starts with open(). It returns a file object that exposes methods for reading, writing, and positioning within the file.

file_object = open(file, mode="r", encoding=None)

• file — the path to the file (a string or a pathlib.Path object).
• mode — how to open the file (see the table below).
• encoding — the text encoding, e.g. "utf-8". Always specify this for text files so your code works the same on every operating system.

File Modes

Always Use with to Open Files

The with statement (a context manager) guarantees the file is closed when the block exits — even if an exception is raised. This prevents resource leaks and ensures buffered writes are flushed to disk.

Open a file safely with with

with open("notes.txt", "r", encoding="utf-8") as f:
    contents = f.read()
# File is automatically closed here

Calling open() without with and forgetting file.close() is a common source of data corruption and "too many open files" errors in long-running programs.

Reading Files

Python provides several ways to read the content of a file.

Read the Entire File with read()

read() returns the complete file content as a single string.

Read an entire file

with open("notes.txt", "r", encoding="utf-8") as f:
    content = f.read()
    print(content)

For large files this loads everything into memory at once, which is fine for small files but inefficient for multi-gigabyte logs.

Read a Fixed Number of Characters

Pass an integer to read(n) to read at most n characters.

Read the first 20 characters

with open("notes.txt", "r", encoding="utf-8") as f:
    chunk = f.read(20)
    print(repr(chunk))

Read One Line at a Time with readline()

readline() returns the next line including its trailing \n, or an empty string at end-of-file.

Read a file line by line with readline()

with open("notes.txt", "r", encoding="utf-8") as f:
    line = f.readline()
    while line:
        print(line, end="")   # line already contains '\n'
        line = f.readline()

Iterate Over Lines (Most Pythonic)

Iterating over a file object directly is the most memory-efficient approach for line-by-line reading.

Iterate over lines

with open("notes.txt", "r", encoding="utf-8") as f:
    for line in f:
        print(line, end="")

Read All Lines into a List with readlines()

readlines() returns a list where each element is one line (with the newline character included).

Read all lines into a list

with open("notes.txt", "r", encoding="utf-8") as f:
    lines = f.readlines()

print(lines[0])   # first line
print(len(lines)) # total number of lines

Use readlines() when you need random access to specific lines by index. For sequential processing, prefer the for line in f pattern.

Writing Files

Write Mode ("w")

Write mode creates the file if it does not exist and truncates (empties) it if it does.

Write text to a file

with open("output.txt", "w", encoding="utf-8") as f:
    f.write("Line one\n")
    f.write("Line two\n")

Write Multiple Lines with writelines()

writelines() accepts an iterable of strings. It does not add newline characters automatically.

Write a list of lines

lines = ["apple\n", "banana\n", "cherry\n"]

with open("fruits.txt", "w", encoding="utf-8") as f:
    f.writelines(lines)

Append Mode ("a")

Append mode moves the write position to the end of the file before each write, so existing content is never overwritten.

Append a log entry

import datetime

entry = f"{datetime.date.today()} — task complete\n"

with open("log.txt", "a", encoding="utf-8") as f:
    f.write(entry)

Each time this script runs it adds a new line to log.txt without touching previous entries.

Exclusive Creation Mode ("x")

Use "x" when you want to create a new file and guarantee you are not clobbering an existing one. Python raises FileExistsError if the file already exists.

Create a file only if it does not exist

try:
    with open("config.txt", "x", encoding="utf-8") as f:
        f.write("[settings]\n")
except FileExistsError:
    print("config.txt already exists — not overwriting.")

File Positions: seek() and tell()

File objects maintain an internal position pointer that advances as you read or write. You can inspect and change this pointer.

• tell() — returns the current byte position.
• seek(offset, whence=0) — moves the pointer. whence=0 (default) is from the start, 1 is from the current position, 2 is from the end.

Rewind to the beginning with seek(0)

with open("notes.txt", "r", encoding="utf-8") as f:
    first_pass = f.read()
    f.seek(0)              # go back to the start
    second_pass = f.read()

print(first_pass == second_pass)  # True

seek() is especially useful in "r+" (read-and-write) mode where you might read a section and then overwrite it in the same open call.

Working with Binary Files

Open a file in binary mode by adding "b" to the mode string ("rb", "wb", "ab"). Binary mode gives you raw bytes instead of strings, which is essential for images, audio, compressed archives, and other non-text data.

Copy a file in binary mode

with open("photo.jpg", "rb") as src:
    data = src.read()

with open("photo_backup.jpg", "wb") as dst:
    dst.write(data)

Do not specify encoding when using binary mode — Python will raise a ValueError if you try.

Error Handling

File operations can fail in predictable ways. Wrapping them in try/except blocks makes your scripts robust.

Handle common file errors

try:
    with open("data.txt", "r", encoding="utf-8") as f:
        content = f.read()
except FileNotFoundError:
    print("Error: the file does not exist.")
except PermissionError:
    print("Error: you do not have permission to read this file.")
except OSError as e:
    print(f"OS error: {e}")

Common exceptions you will encounter:

The Modern pathlib Alternative

Python 3.4 introduced pathlib.Path, an object-oriented approach to file system paths. Path objects work seamlessly with open() and also expose their own read_text() / write_text() convenience methods.

Read a file with pathlib

from pathlib import Path

content = Path("notes.txt").read_text(encoding="utf-8")
print(content)

Write a file with pathlib

from pathlib import Path

Path("output.txt").write_text("Hello, world!\n", encoding="utf-8")

read_text() and write_text() open and close the file for you, making one-shot reads and writes very concise. Use open() with a with block when you need finer control — for example, reading a file in chunks or using seek().

Renaming and Moving Files

To rename or move a file, use os.rename() for a same-filesystem rename, or shutil.move() when you need to move across filesystems.

Rename a file with os.rename()

import os

os.rename("old_name.txt", "new_name.txt")

Move a file with shutil.move()

import shutil

shutil.move("report.txt", "archive/report.txt")

shutil.move() works even when the source and destination are on different drives; os.rename() raises OSError in that case.

Checking Whether a File Exists

Before opening a file for reading, you may want to confirm it exists. Use os.path.exists() or the pathlib equivalent.

Check existence with os.path

import os

if os.path.exists("data.txt"):
    print("File found.")
else:
    print("File not found.")

Check existence with pathlib

from pathlib import Path

p = Path("data.txt")
if p.exists():
    print("File found.")

Note that os.path.exists() and Path.exists() can return stale results in multi-threaded or multi-process programs. In those cases, prefer to just open() the file and catch FileNotFoundError.

Summary of Key Functions

Related Chapters

• Python Read Files — deep dive into every reading technique
• Python Write / Create Files — writing, creating, and best practices
• Python Delete Files — deleting files and directories safely
• Python Try Except — handling exceptions in Python
• Python With Statement — how context managers work