Python Regular Expressions (RegEx) — Complete Guide

Regular expressions (regex) let you search, extract, and replace text based on flexible patterns rather than exact strings. Python's built-in re module provides everything you need. This chapter covers the full regex toolkit: syntax, special sequences, quantifiers, groups, lookahead/lookbehind, flags, and every key re function — with correct, runnable examples throughout.

Why Use Regular Expressions?

Plain string methods (str.find(), str.replace(), str.split()) work well for fixed text. Regular expressions shine when the pattern varies:

Validate that a string looks like an email address or phone number.
Extract all dates from a document, regardless of exact format.
Strip HTML tags from a string.
Replace multiple different whitespace sequences with a single space.

When the task involves describing a shape rather than a fixed value, reach for re.

Raw Strings

Regex patterns are almost always written as raw strings (r'...'). Raw strings treat backslashes as literal characters, which matters because regex uses many backslash sequences (\d, \w, \s, \b). Without the r prefix you would need double backslashes everywhere:

import re

# Both patterns are identical — raw string is easier to read
re.findall(r'\d+', 'abc 123')   # raw:    r'\d+'
re.findall('\\d+', 'abc 123')   # normal: '\\d+'

Use raw strings for every regex pattern — it is the universal convention.

Basic Syntax: Metacharacters

Metacharacters are characters with special meaning inside a pattern. Literal characters match themselves exactly.

Metacharacter	Meaning
`.`	Any character except a newline
`^`	Start of the string (or line in `MULTILINE` mode)
`$`	End of the string (or line in `MULTILINE` mode)
`*`	Zero or more of the preceding element
`+`	One or more of the preceding element
`?`	Zero or one of the preceding element
`{n}`	Exactly n repetitions
`{n,m}`	Between n and m repetitions
`[...]`	Character class — any one character listed inside
`[^...]`	Negated class — any character not listed
`\|`	Alternation — either the left or right expression
`()`	Capturing group
`\`	Escape a metacharacter, or start a special sequence

To match a literal metacharacter such as . or *, escape it with a backslash: \. matches a real dot.

Special Sequences

Special sequences are shorthand character classes that appear frequently in real-world patterns.

Sequence	Matches
`\d`	Any digit — same as `[0-9]`
`\D`	Any non-digit
`\w`	Word character: letters, digits, underscore
`\W`	Non-word character
`\s`	Whitespace: space, tab, newline
`\S`	Non-whitespace
`\b`	Word boundary (zero-width)
`\B`	Non-boundary

import re

print(re.findall(r'\d+', 'I have 3 cats and 12 dogs'))
# ['3', '12']

print(re.findall(r'\w+', 'hello_world 123'))
# ['hello_world', '123']

print(re.findall(r'\bPython\b', 'Python Pythonista Python3'))
# ['Python']  — word boundary prevents partial matches

\b is particularly useful: it matches the position between a word character and a non-word character, so \bPython\b matches the standalone word "Python" but not "Pythonista" or "Python3".

Quantifiers

Quantifiers control how many times the preceding element must match.

import re

print(re.findall(r'a*', 'baaa'))   # ['', 'aaa', '']
print(re.findall(r'a+', 'baaa'))   # ['aaa']
print(re.findall(r'a?', 'baaa'))   # ['', 'a', 'a', 'a', '']
print(re.findall(r'a{3}', 'baaa')) # ['aaa']

Greedy vs Lazy Matching

By default, quantifiers are greedy — they match as much text as possible. Add ? after the quantifier to make it lazy (match as little as possible).

import re

html = '<b>bold</b> and <i>italic</i>'

print(re.findall(r'<.*>', html))
# ['<b>bold</b> and <i>italic</i>']  — greedy: matches from first < to last >

print(re.findall(r'<.*?>', html))
# ['<b>', '</b>', '<i>', '</i>']  — lazy: matches each individual tag

Lazy quantifiers are essential when parsing structured text like HTML or JSON fragments.

Character Classes

A character class [...] matches any one character from the listed set. Use - for ranges and ^ at the start to negate the class.

import re

print(re.findall(r'[aeiou]', 'hello world'))
# ['e', 'o', 'o']

print(re.findall(r'[0-9]', 'a1b2c3'))
# ['1', '2', '3']

print(re.findall(r'[^aeiou\s]+', 'hello world'))
# ['h', 'll', 'w', 'rld']  — consonants only (not vowels, not spaces)

Common ready-made ranges: [a-z] lowercase letters, [A-Z] uppercase, [0-9] digits, [a-zA-Z0-9] alphanumeric.

Anchors

Anchors do not consume characters — they assert a position in the string.

import re

print(re.findall(r'^Python', 'Python is great'))
# ['Python']  — matches only if 'Python' is at the start

print(re.findall(r'great$', 'Python is great'))
# ['great']  — matches only if 'great' is at the end

print(re.findall(r'^Python', 'Learn Python'))
# []  — 'Python' is not at the start of this string

See the re.MULTILINE flag later in this chapter for applying ^ and $ per line rather than per string.

Alternation

The pipe | works like a logical OR between two expressions.

import re

print(re.findall(r'cat|dog', 'I have a cat and a dog'))
# ['cat', 'dog']

print(re.findall(r'colou?r|colour', 'color and colour'))
# ['color', 'colour']

Capturing Groups

Parentheses () create a capturing group. Groups let you extract sub-parts of a match. re.search() returns a match object; call .group(n) or .groups() on it.

import re

match = re.search(r'(\d{4})-(\d{2})-(\d{2})', '2023-10-05')
if match:
    print(match.group(0))   # '2023-10-05'  — entire match
    print(match.group(1))   # '2023'
    print(match.groups())   # ('2023', '10', '05')

Named Groups

Name a group with (?P<name>...) to access it by name instead of position. This makes patterns much easier to maintain.

import re

match = re.search(
    r'(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})',
    '2023-10-05'
)
if match:
    print(match.group('year'))   # '2023'
    print(match.groupdict())     # {'year': '2023', 'month': '10', 'day': '05'}

Non-Capturing Groups

Use (?:...) when you need to group but do not need to capture the value.

import re

print(re.findall(r'(?:Mr|Mrs|Ms)\.? \w+', 'Mr. Smith and Mrs. Jones'))
# ['Mr. Smith', 'Mrs. Jones']

Lookahead and Lookbehind

Lookahead ((?=...)) and lookbehind ((?<=...)) assert that something follows or precedes the match, without including it in the result. They are zero-width: they consume no characters.

import re

# Positive lookahead — find numbers followed by 'dollars'
print(re.findall(r'\d+(?= dollars)', '100 dollars and 200 euros'))
# ['100']

# Negative lookahead — find numbers NOT followed by 'dollars'
print(re.findall(r'\b\d+\b(?! dollars)', '100 dollars and 200 euros'))
# ['200']

# Positive lookbehind — extract domain from email addresses
emails = 'Contact [email protected] or [email protected]'
print(re.findall(r'(?<=@)\w+\.\w+', emails))
# ['example.com', 'test.org']

Lookbehind patterns must have a fixed width in Python — you cannot use * or + inside one.

The `re` Module: Core Functions

`re.search()` — Find the First Match

Returns a match object for the first location where the pattern matches, or None if there is no match.

import re

text = 'apple banana apple'
match = re.search(r'banana', text)
if match:
    print(match.group())   # 'banana'
    print(match.span())    # (6, 12)

`re.match()` — Match at the Start Only

Like re.search(), but the pattern must match at the beginning of the string.

import re

print(bool(re.match(r'\d+', 'abc123')))   # False — no digit at start
print(bool(re.search(r'\d+', 'abc123')))  # True  — digit found anywhere

Use re.search() when you want to find a pattern anywhere; use re.match() when the pattern must appear at the start.

`re.fullmatch()` — Match the Entire String

The pattern must match the whole string from start to end.

import re

print(bool(re.fullmatch(r'\d{5}', '12345')))   # True  — exactly 5 digits
print(bool(re.fullmatch(r'\d{5}', '123456')))  # False — too long
print(bool(re.fullmatch(r'\d{5}', '1234X')))   # False — non-digit present

re.fullmatch() is ideal for input validation (postal codes, phone numbers, etc.).

`re.findall()` — All Non-Overlapping Matches

Returns a list of all matches. If the pattern has groups, returns a list of tuples.

import re

print(re.findall(r'\d+', 'abc 123 def 456'))
# ['123', '456']

# With a group — returns list of group values
print(re.findall(r'(\w+)@(\w+\.\w+)', '[email protected] [email protected]'))
# [('alice', 'example.com'), ('bob', 'test.org')]

`re.finditer()` — Iterator of Match Objects

Like re.findall(), but yields match objects one at a time. Useful for large texts or when you need span information.

import re

for m in re.finditer(r'\d+', 'abc 123 def 456'):
    print(m.group(), m.start(), m.end())
# 123 4 7
# 456 12 15

`re.sub()` — Replace Matches

Replaces every occurrence of the pattern with a replacement string or the return value of a function.

import re

text = 'apple banana apple'
print(re.sub(r'apple', 'orange', text))
# 'orange banana orange'

# Limit replacements
print(re.sub(r'apple', 'orange', text, count=1))
# 'orange banana apple'

# Backreferences in replacement — reformat a date
print(re.sub(r'(\d{4})-(\d{2})-(\d{2})', r'\3/\2/\1', '2023-10-05'))
# '05/10/2023'

`re.split()` — Split by a Pattern

Splits the string at each occurrence of the pattern.

import re

print(re.split(r',\s*', 'apple, banana, cherry, date'))
# ['apple', 'banana', 'cherry', 'date']

# Limit the number of splits
print(re.split(r',\s*', 'apple, banana, cherry, date', maxsplit=2))
# ['apple', 'banana', 'cherry, date']

Compiling Patterns with `re.compile()`

When you use the same pattern many times, compile it once with re.compile() to improve performance. The resulting pattern object has all the same methods (findall, search, sub, etc.).

import re

# Compile once
digit_pattern = re.compile(r'\d+')

# Reuse many times
print(digit_pattern.findall('abc 123 def 456'))   # ['123', '456']
print(digit_pattern.sub('NUM', 'abc 123 def 456')) # 'abc NUM def NUM'
print(digit_pattern.search('xyz 99'))              # <re.Match object ...>

Compilation is especially valuable inside loops or functions called frequently.

Flags

Flags modify matching behavior. Pass them as the last argument to any re function, or include them when calling re.compile(). Multiple flags can be combined with |.

Flag	Short form	Effect
`re.IGNORECASE`	`re.I`	Case-insensitive matching
`re.MULTILINE`	`re.M`	`^` and `$` match start/end of each line
`re.DOTALL`	`re.S`	`.` matches newlines too
`re.VERBOSE`	`re.X`	Allow whitespace and comments inside the pattern

import re

# IGNORECASE
print(re.findall(r'hello', 'Hello HELLO hello', re.IGNORECASE))
# ['Hello', 'HELLO', 'hello']

# MULTILINE — ^ matches the start of each line
text = 'first line\nsecond line\nthird line'
print(re.findall(r'^\w+', text, re.MULTILINE))
# ['first', 'second', 'third']

# DOTALL — . matches newline characters
print(re.findall(r'<.*?>', '<div>\n<p>text</p>\n</div>', re.DOTALL))
# ['<div>', '<p>', '</p>', '</div>']

# VERBOSE — write readable patterns with comments
date_pattern = re.compile(r'''
    (?P<year>\d{4})   # four-digit year
    -
    (?P<month>\d{2})  # two-digit month
    -
    (?P<day>\d{2})    # two-digit day
''', re.VERBOSE)
print(date_pattern.search('2023-10-05').groupdict())
# {'year': '2023', 'month': '10', 'day': '05'}

Escaping User Input with `re.escape()`

If you build a pattern from user-supplied text, always escape it first to prevent unintended metacharacter interpretation.

import re

user_input = 'hello.world'
# Without escaping, '.' matches any character
# With escaping, '\.' matches a literal dot
safe_pattern = re.escape(user_input)
print(safe_pattern)                                    # 'hello\\.world'
print(bool(re.search(safe_pattern, 'say hello.world')))  # True
print(bool(re.search(safe_pattern, 'say helloXworld')))  # False

Practical Examples

Validate an Email Address

import re

def is_valid_email(email):
    pattern = r'^[\w.+-]+@[\w-]+\.[a-zA-Z]{2,}$'
    return bool(re.fullmatch(pattern, email))

print(is_valid_email('[email protected]'))   # True
print(is_valid_email('bad-email@'))         # False

Extract All URLs from Text

import re

text = 'Visit https://www.example.com or http://test.org for details.'
urls = re.findall(r'https?://[\w./-]+', text)
print(urls)
# ['https://www.example.com', 'http://test.org']

Remove Extra Whitespace

import re

messy = '  hello    world   Python  '
clean = re.sub(r'\s+', ' ', messy).strip()
print(clean)  # 'hello world Python'

Parse a Log Line

import re

log = '2023-10-05 14:32:11 ERROR Failed to connect to database'
pattern = re.compile(
    r'(?P<date>\d{4}-\d{2}-\d{2}) '
    r'(?P<time>\d{2}:\d{2}:\d{2}) '
    r'(?P<level>\w+) '
    r'(?P<message>.+)'
)
m = pattern.match(log)
if m:
    print(m.group('level'))    # 'ERROR'
    print(m.group('message'))  # 'Failed to connect to database'

Common Gotchas

re.match() vs re.search() — re.match() only checks the start of the string. New users often expect it to search anywhere and are confused when it returns None.

Forgetting raw strings — '\d' in a normal Python string is just 'd' with an unrecognized escape (or a SyntaxWarning in Python 3.12+). Always write r'\d'.

Greedy matches swallowing too much — If a .* pattern grabs more than you intended, switch to .*? (lazy).

Special characters in replacement strings — In re.sub(), backslash sequences like \1 in the replacement refer to captured groups. To include a literal backslash in the replacement, write \\.

re.findall() with groups — When your pattern contains groups, re.findall() returns the groups, not the full match. Use a non-capturing group (?:...) if you want the full match.

Quick Reference

Task	Function
Find first match	`re.search(pattern, text)`
Find all matches	`re.findall(pattern, text)`
Iterate matches	`re.finditer(pattern, text)`
Match at start	`re.match(pattern, text)`
Match whole string	`re.fullmatch(pattern, text)`
Replace	`re.sub(pattern, repl, text)`
Split	`re.split(pattern, text)`
Compile for reuse	`re.compile(pattern)`
Escape user input	`re.escape(text)`

Python Strings — string methods that complement regex for simpler text tasks.
Modify Strings — built-in methods like replace() and split() that avoid regex when patterns are fixed.
Python File Handling — reading files line by line to apply regex at scale.
Python Try/Except — handling errors that arise when regex patterns are compiled from user input.
Python Functions — wrapping regex logic in reusable functions.

Practice

What does the 're' module in Python provide?

Functions for generating random numbers.Functions for interacting with the operating system.Functions for working with regular expressions.Functions for mathematical operations.