W3docs
en
Log in

Python Working with APIs (requests)

Learn to use Python's requests library to make GET and POST calls, send headers and query parameters, handle JSON responses, and manage errors.

The requests library is the standard way to make HTTP calls from Python. It wraps Python's lower-level urllib in a clean API, so fetching a web page or calling a REST API takes one line instead of ten. This chapter covers everything you need: installing requests, making GET and POST calls, sending headers and query parameters, working with JSON responses, uploading files, using sessions, and handling errors robustly.

Installation

requests is not part of the standard library, so you install it with pip:

pip install requests

If you are working inside a virtual environment (recommended), activate it first so the package is scoped to your project. After installation, verify it works:

import requests
print(requests.__version__)   # e.g. 2.32.3

Making a GET request

requests.get() sends an HTTP GET request and returns a Response object. This is the most common operation — used for fetching data from APIs, web pages, and files.

import requests

response = requests.get("https://jsonplaceholder.typicode.com/todos/1")

print(response.status_code)   # 200
print(response.url)           # https://jsonplaceholder.typicode.com/todos/1
print(response.text)          # raw response body as a string

Expected output:

200
https://jsonplaceholder.typicode.com/todos/1
{"userId": 1, "id": 1, "title": "delectus aut autem", "completed": false}

Reading the response as JSON

Most modern APIs return JSON. Call .json() on the response instead of parsing .text manually — it calls json.loads() for you and returns a Python dict or list.

import requests

response = requests.get("https://jsonplaceholder.typicode.com/todos/1")
data = response.json()

print(data["title"])      # delectus aut autem
print(data["completed"])  # False

See the Python JSON chapter for details on how Python objects map to JSON types.

Sending query parameters

Query parameters are the key-value pairs after the ? in a URL, like ?q=python&page=2. Pass them as a dict to the params argument — requests URL-encodes and appends them automatically.

import requests

params = {
    "q": "python requests",
    "page": 1,
    "per_page": 5,
}

response = requests.get("https://httpbin.org/get", params=params)

# requests builds the full URL for you
print(response.url)
# https://httpbin.org/get?q=python+requests&page=1&per_page=5

Always use params= rather than building the URL by hand — it handles special characters and encoding correctly.

Sending request headers

Headers carry metadata: authentication tokens, content-type preferences, API keys, and more. Pass them as a dict to headers=:

import requests

headers = {
    "Accept": "application/json",
    "Authorization": "Bearer my-api-token",
    "User-Agent": "MyApp/1.0",
}

response = requests.get("https://httpbin.org/headers", headers=headers)
print(response.json())

Common headers you will send:

HeaderPurpose
AuthorizationAuthentication token (Bearer, Basic, etc.)
Content-TypeFormat of the request body (e.g. application/json)
AcceptFormat you want back from the server
User-AgentIdentifies your client to the server
X-API-KeyAPI key in a custom header (varies by service)

Making a POST request

requests.post() sends data to the server — used to create resources, submit forms, or call actions.

Sending JSON

Pass a Python dict to json=. The library serializes it and sets the Content-Type: application/json header automatically:

import requests

payload = {
    "title": "Buy groceries",
    "completed": False,
    "userId": 1,
}

response = requests.post(
    "https://jsonplaceholder.typicode.com/todos",
    json=payload,
)

print(response.status_code)   # 201 Created
print(response.json())

Expected output:

201
{'title': 'Buy groceries', 'completed': False, 'userId': 1, 'id': 201}

Sending form data

Some APIs or HTML forms expect application/x-www-form-urlencoded data. Use data= instead of json=:

import requests

form_data = {
    "username": "alice",
    "password": "secret",
}

response = requests.post("https://httpbin.org/post", data=form_data)
print(response.status_code)

Sending files (multipart upload)

To upload a file, open it in binary mode and pass it through files=:

import requests

with open("report.pdf", "rb") as f:
    response = requests.post(
        "https://httpbin.org/post",
        files={"file": f},
    )

print(response.status_code)

requests encodes the upload as multipart/form-data, which is what most file-upload endpoints expect.

Other HTTP methods

REST APIs use different HTTP verbs for different operations. requests provides one function per method:

import requests

base = "https://jsonplaceholder.typicode.com/todos/1"

# Update a resource (replace entirely)
response = requests.put(base, json={"title": "Updated", "completed": True, "userId": 1})
print(response.status_code)   # 200

# Partial update
response = requests.patch(base, json={"completed": True})
print(response.status_code)   # 200

# Delete a resource
response = requests.delete(base)
print(response.status_code)   # 200

Error handling

Checking status codes

The HTTP status code tells you whether the request succeeded. The most important groups:

RangeMeaning
2xxSuccess (200 OK, 201 Created, 204 No Content)
3xxRedirect (handled automatically by requests)
4xxClient error (400 Bad Request, 401 Unauthorized, 404 Not Found)
5xxServer error (500 Internal Server Error, 503 Service Unavailable)

raise_for_status()

Calling .raise_for_status() on a response raises an HTTPError exception automatically if the status code is 4xx or 5xx. This is the cleanest way to fail fast on bad responses:

import requests

response = requests.get("https://jsonplaceholder.typicode.com/todos/99999")

try:
    response.raise_for_status()
    data = response.json()
    print(data)
except requests.exceptions.HTTPError as err:
    print(f"HTTP error: {err}")

Without raise_for_status(), a 404 response looks like a success — your code reads an error body and silently processes it.

Handling network errors

Network-level failures (DNS lookup failure, connection refused, timeout) raise requests.exceptions.ConnectionError or requests.exceptions.Timeout. Catch both with the base requests.exceptions.RequestException:

import requests

try:
    response = requests.get("https://api.example.com/data", timeout=5)
    response.raise_for_status()
    data = response.json()
except requests.exceptions.Timeout:
    print("The request timed out — server took too long to respond.")
except requests.exceptions.ConnectionError:
    print("Could not connect — check your network or the URL.")
except requests.exceptions.HTTPError as err:
    print(f"HTTP error {response.status_code}: {err}")
except requests.exceptions.RequestException as err:
    print(f"Unexpected error: {err}")

This pattern covers the full exception hierarchy: timeout, connection, HTTP error, and the catch-all base class. See Python try/except for a deeper look at exception handling in Python.

Always set a timeout

By default requests will wait forever if the server never responds. Always pass timeout= to avoid hanging programs:

# timeout=(connect_timeout, read_timeout) in seconds
response = requests.get("https://api.example.com/data", timeout=(3, 10))

The tuple form sets the connection timeout and the read timeout separately. A read timeout of 10 seconds means "wait up to 10 seconds between bytes after the connection is established."

Using sessions

A requests.Session object persists settings — headers, cookies, authentication — across multiple requests to the same host. It also reuses the underlying TCP connection (connection pooling), which is faster than creating a new connection for every call.

import requests

with requests.Session() as session:
    # Set headers once — every request in this session will include them
    session.headers.update({
        "Authorization": "Bearer my-api-token",
        "Accept": "application/json",
    })

    # All requests reuse the connection and headers
    r1 = session.get("https://api.example.com/users")
    r2 = session.get("https://api.example.com/posts")
    r3 = session.post("https://api.example.com/todos", json={"title": "New"})

    print(r1.status_code, r2.status_code, r3.status_code)

Use a session whenever you are making more than one request to the same server.

Inspecting the response

The Response object exposes everything the server sent back:

import requests

response = requests.get("https://httpbin.org/get")

print(response.status_code)      # 200
print(response.reason)           # OK
print(response.headers)          # dict of response headers
print(response.headers["Content-Type"])  # application/json
print(response.encoding)         # utf-8
print(response.elapsed)          # how long the request took
print(response.url)              # final URL (after redirects)

For binary content (images, PDFs), use response.content (returns bytes) instead of response.text:

import requests

response = requests.get("https://httpbin.org/image/png")
with open("image.png", "wb") as f:
    f.write(response.content)

Authentication

HTTP Basic Auth

Pass a (username, password) tuple to auth=:

import requests

response = requests.get(
    "https://httpbin.org/basic-auth/alice/secret",
    auth=("alice", "secret"),
)
print(response.status_code)   # 200

Bearer token (API keys)

Most modern APIs use a bearer token in the Authorization header:

import requests

headers = {"Authorization": "Bearer eyJhbGciOiJIUzI1NiIs..."}
response = requests.get("https://api.example.com/me", headers=headers)

Never hard-code tokens in source files. Load them from environment variables or a secrets manager:

import os
import requests

token = os.environ["API_TOKEN"]
headers = {"Authorization": f"Bearer {token}"}
response = requests.get("https://api.example.com/me", headers=headers)

Real-world example — GitHub API

This example demonstrates a complete pattern: session, bearer auth, pagination, error handling, and JSON parsing:

import os
import requests

GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN", "")
BASE_URL = "https://api.github.com"

with requests.Session() as session:
    session.headers.update({
        "Authorization": f"Bearer {GITHUB_TOKEN}",
        "Accept": "application/vnd.github+json",
        "X-GitHub-Api-Version": "2022-11-28",
    })

    try:
        # Fetch the first page of public repos for a user
        response = session.get(
            f"{BASE_URL}/users/torvalds/repos",
            params={"per_page": 5, "sort": "updated"},
            timeout=10,
        )
        response.raise_for_status()
        repos = response.json()

        for repo in repos:
            print(f"{repo['name']:40s} ★ {repo['stargazers_count']}")

    except requests.exceptions.HTTPError as err:
        print(f"GitHub API error: {err}")
    except requests.exceptions.RequestException as err:
        print(f"Network error: {err}")

This pattern — session with shared headers, raise_for_status(), scoped try/except — is the production-ready approach for any API client you write.

Concurrent requests with asyncio

For programs that call many endpoints at once, the synchronous requests library blocks on each call. Switch to aiohttp (the async equivalent) and combine it with Python's asyncio module:

import asyncio
import aiohttp

async def fetch(session, url):
    async with session.get(url) as response:
        return await response.json()

async def main():
    urls = [
        "https://jsonplaceholder.typicode.com/todos/1",
        "https://jsonplaceholder.typicode.com/todos/2",
        "https://jsonplaceholder.typicode.com/todos/3",
    ]
    async with aiohttp.ClientSession() as session:
        results = await asyncio.gather(*[fetch(session, u) for u in urls])
    for r in results:
        print(r["title"])

asyncio.run(main())

See the Python asyncio chapter to understand how async/await works before adopting this pattern.

Quick reference

TaskCode
GET requestrequests.get(url)
GET with paramsrequests.get(url, params={"key": "val"})
GET with headersrequests.get(url, headers={"Authorization": "Bearer token"})
POST JSON bodyrequests.post(url, json={"key": "val"})
POST form datarequests.post(url, data={"key": "val"})
Upload a filerequests.post(url, files={"file": open("f.pdf", "rb")})
PUT / PATCH / DELETErequests.put/patch/delete(url, json=...)
Check statusresponse.status_code
Raise on 4xx/5xxresponse.raise_for_status()
Parse JSON bodyresponse.json()
Raw text bodyresponse.text
Raw bytes bodyresponse.content
Set timeoutrequests.get(url, timeout=5)
Reuse connectionwith requests.Session() as s: ...

Practice

Practice
Which argument sends a Python dict as a JSON body in a POST request?
Which argument sends a Python dict as a JSON body in a POST request?
Was this page helpful?