Python os and sys Modules

The os and sys modules are two of Python's most essential standard-library tools. os bridges Python and the underlying operating system — letting you navigate directories, inspect and set environment variables, and manipulate files and paths. sys bridges Python and the interpreter itself — exposing the argument list, the module search path, and hooks for controlling interpreter exit. Together they cover almost everything a script needs to interact with the environment it runs in.

The os Module

Import os at the top of your script:

import os

No installation is needed; it ships with every Python distribution.

Working with Directories

Get and Change the Current Working Directory

os.getcwd() returns the absolute path of the directory where your script is running. os.chdir() changes it.

import os

# Print the current directory
print(os.getcwd())
# Example output: /Users/alice/projects

# Change to a different directory
os.chdir("/tmp")
print(os.getcwd())
# Output: /tmp

List Directory Contents

os.listdir(path) returns a list of all entries (files and subdirectories) in path. It does not recurse into subdirectories.

import os

entries = os.listdir(".")   # "." means current directory
for entry in sorted(entries):
    print(entry)

To distinguish files from directories use os.path.isfile() and os.path.isdir():

import os

for entry in os.listdir("."):
    if os.path.isdir(entry):
        print(f"[DIR]  {entry}")
    else:
        print(f"[FILE] {entry}")

Create and Remove Directories

import os

# Create a single directory
os.mkdir("reports")

# Create nested directories in one call
os.makedirs("data/2024/january", exist_ok=True)
# exist_ok=True prevents an error if the directory already exists

# Remove an empty directory
os.rmdir("reports")

# Remove a full directory tree
import shutil
shutil.rmtree("data")

The exist_ok=True parameter in os.makedirs() is very useful in scripts that may run more than once — without it, a second run raises FileExistsError.

Walk a Directory Tree

os.walk(top) generates a three-tuple (dirpath, dirnames, filenames) for every directory in the tree rooted at top. It is the standard way to recurse through a folder structure.

import os

for dirpath, dirnames, filenames in os.walk("project"):
    level = dirpath.count(os.sep)
    indent = "  " * level
    print(f"{indent}{os.path.basename(dirpath)}/")
    for filename in filenames:
        print(f"{indent}  {filename}")

Working with File Paths

Python's os.path sub-module contains portable path-manipulation utilities that work correctly on Windows, macOS, and Linux.

Join Path Components

os.path.join() combines parts of a path using the correct separator for the current OS.

import os

base = "/home/alice"
project = "myapp"
filename = "config.json"

full_path = os.path.join(base, project, filename)
print(full_path)
# Output: /home/alice/myapp/config.json

Never build paths with string concatenation like base + "/" + filename — it breaks on Windows. Always use os.path.join().

Split a Path

import os

path = "/home/alice/myapp/config.json"

print(os.path.dirname(path))   # /home/alice/myapp
print(os.path.basename(path))  # config.json
print(os.path.split(path))     # ('/home/alice/myapp', 'config.json')
print(os.path.splitext(path))  # ('/home/alice/myapp/config', '.json')

os.path.splitext() is convenient when you need to strip or change a file extension.

Check Whether a Path Exists

import os

print(os.path.exists("/tmp"))      # True (usually)
print(os.path.isfile("/tmp"))      # False — it is a directory
print(os.path.isdir("/tmp"))       # True
print(os.path.isabs("/tmp"))       # True — it is an absolute path

Get the Absolute Path

os.path.abspath() resolves relative paths against the current working directory:

import os

print(os.path.abspath("config.json"))
# Example output: /home/alice/myapp/config.json

This is useful when you need to store or log a path that must remain valid even if the working directory changes later.

The __file__ Variable and Script-Relative Paths

A common gotcha: a script that opens "data.csv" works when you run it from its own directory but breaks from a different one. The fix is to build the path relative to the script file itself:

import os

# Directory that contains *this* script
HERE = os.path.dirname(os.path.abspath(__file__))

data_file = os.path.join(HERE, "data.csv")
with open(data_file, encoding="utf-8") as f:
    content = f.read()

This technique makes scripts portable regardless of which directory you launch them from.

Environment Variables

Environment variables store configuration that lives outside the source code — database URLs, API keys, feature flags, and so on. The os.environ mapping gives you read and write access to the current process's environment.

Read an Environment Variable

import os

# Returns the value or None if not set
home = os.environ.get("HOME")
print(home)
# Example output: /home/alice

# Raise KeyError if not set (useful to fail fast on missing config)
path = os.environ["PATH"]

Prefer os.environ.get(key) over os.environ[key] unless the variable is strictly required and the program should crash without it.

Read with a Default Value

import os

debug = os.environ.get("DEBUG", "false")
port = int(os.environ.get("PORT", "8080"))

print(f"debug={debug}, port={port}")
# Output: debug=false, port=8080

Set and Delete Environment Variables

import os

# Set a variable — affects only the current process and its children
os.environ["MY_APP_ENV"] = "production"

# Remove a variable
os.environ.pop("MY_APP_ENV", None)   # None prevents KeyError if not present

Setting os.environ values does not persist after the process exits. To set permanent environment variables, modify your shell profile (~/.bashrc, ~/.zshrc) or use an .env file loaded by a library such as python-dotenv.

List All Environment Variables

import os

for key, value in sorted(os.environ.items()):
    print(f"{key}={value}")

Running Shell Commands with os.system() and subprocess

os.system(command) runs a shell command and returns its exit code, but gives you no way to capture its output. For anything beyond a quick fire-and-forget call, use the subprocess module instead.

import os
import subprocess

# Quick way — exit code only
exit_code = os.system("echo hello")
print("exit code:", exit_code)   # 0 means success

# Better way — capture output
result = subprocess.run(
    ["echo", "hello"],
    capture_output=True,
    text=True,
)
print(result.stdout.strip())   # hello

subprocess.run() is more powerful and safer than os.system() because it avoids the shell interpreter and lets you capture stdout, stderr, and the return code as Python objects.

Useful os Utilities

The sys Module

sys exposes information about the Python interpreter and gives you hooks to control its behaviour.

import sys

Command-Line Arguments with sys.argv

sys.argv is a list of strings. sys.argv[0] is the script name; subsequent elements are the arguments passed on the command line.

Suppose you save this as greet.py and run python greet.py Alice 42:

import sys

script_name = sys.argv[0]   # 'greet.py'
name = sys.argv[1]          # 'Alice'
age = sys.argv[2]           # '42' — always a string

print(f"Hello, {name}! You are {age} years old.")
# Output: Hello, Alice! You are 42 years old.

Always validate sys.argv before accessing indices — an IndexError will crash the script if the user forgets an argument:

import sys

if len(sys.argv) != 3:
    print(f"Usage: python {sys.argv[0]} <name> <age>")
    sys.exit(1)

name = sys.argv[1]
age = sys.argv[2]
print(f"Hello, {name}! You are {age} years old.")

For complex argument parsing, prefer the argparse module in the standard library — it generates --help output automatically.

Exit the Interpreter with sys.exit()

sys.exit(code) raises SystemExit and terminates the interpreter. By convention, exit code 0 means success; any non-zero value signals an error.

import sys

answer = input("Continue? (y/n): ")
if answer.lower() != "y":
    print("Goodbye!")
    sys.exit(0)

print("Continuing...")

You can catch SystemExit in a try/except block if you need to run cleanup before the program exits, but usually the with statement (for file handles, network connections, etc.) handles cleanup automatically.

Python Version Information

import sys

print(sys.version)
# Example: 3.10.15 (main, ...) [GCC 11.4.0]

print(sys.version_info)
# sys.version_info(major=3, minor=10, micro=15, ...)

# Guard against running on an unsupported Python version
if sys.version_info < (3, 8):
    sys.exit("This script requires Python 3.8 or later.")

sys.version_info is a named tuple, so you can compare it directly to a tuple of integers.

The Module Search Path (sys.path)

When you write import mymodule, Python searches each directory in sys.path in order until it finds a matching file. The list starts with the script's directory (or an empty string for interactive sessions), followed by PYTHONPATH entries, and then the standard library and site-packages.

import sys

for p in sys.path:
    print(p)

You can append to sys.path at runtime to import modules from non-standard locations:

import sys
import os

# Add a sibling directory to the search path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "libs"))

import mymodule   # now found in ./libs/mymodule.py

Modifying sys.path is a quick fix for local development, but for distributable packages use pip and a proper pyproject.toml instead.

Standard Streams

sys.stdin, sys.stdout, and sys.stderr are file-like objects connected to the three standard streams. You can redirect them to capture or suppress output.

import sys

# Write to stdout (same as print, but more explicit)
sys.stdout.write("Hello, stdout\n")

# Write to stderr (errors and diagnostics)
sys.stderr.write("Warning: something looks off\n")

A common pattern in scripts is to redirect sys.stdout to a file to capture all print() output:

import sys

with open("output.log", "w", encoding="utf-8") as log:
    original_stdout = sys.stdout
    sys.stdout = log
    print("This goes to the log file.")
    sys.stdout = original_stdout

print("This goes back to the terminal.")

Useful sys Attributes

Combining os and sys in Practice

Real scripts often use both modules together. Here is a small but realistic example: a script that scans a directory for .log files and prints a summary.

import os
import sys

def summarize_logs(directory):
    if not os.path.isdir(directory):
        sys.stderr.write(f"Error: '{directory}' is not a directory.\n")
        sys.exit(1)

    log_files = [
        f for f in os.listdir(directory)
        if f.endswith(".log") and os.path.isfile(os.path.join(directory, f))
    ]

    if not log_files:
        print("No .log files found.")
        return

    print(f"Found {len(log_files)} log file(s) in '{directory}':")
    for name in sorted(log_files):
        full_path = os.path.join(directory, name)
        size = os.path.getsize(full_path)
        print(f"  {name}  ({size} bytes)")

if len(sys.argv) != 2:
    print(f"Usage: python {sys.argv[0]} <directory>")
    sys.exit(1)

summarize_logs(sys.argv[1])

Run it as python summarize.py /var/log and it prints the name and byte-size of every .log file in that directory.

os vs pathlib — Which Should You Use?

Python 3.4 introduced pathlib.Path, an object-oriented alternative to os.path. Both approaches are correct; the choice is mostly a style preference.

pathlib tends to produce more readable code for path-heavy scripts; os.path is familiar and supported everywhere Python 3 runs. The sys module has no pathlib equivalent — it is always import sys.

Practice

Related Chapters

• Python Modules — creating and importing your own modules
• Python File Handling — reading and writing files
• Python Try Except — handling errors from file and OS operations
• Python Variables — variable scope and naming
• Python Scope — how Python resolves names across scopes