How do I remove duplicates from a list, while preserving order?

There are several ways to remove duplicates from a list while preserving order. One reliable method in Python 3.7+ is to use dict.fromkeys(), as standard dictionaries preserve insertion order. Here's an example:

Remove duplicates from a list, while preserving order in Python using dict.fromkeys()

original_list = [1, 2, 3, 4, 2, 5, 3, 6]

# Use dict.fromkeys() to remove duplicates while preserving order
no_duplicates_list = list(dict.fromkeys(original_list))

print(no_duplicates_list)
# Output: [1, 2, 3, 4, 5, 6]

<div class="alert alert-info flex not-prose"> <span class="hidden md:block">Watch a video course </span> Python - The Practical Guide</div>

Another way is to iterate through the list and add items to a new list only if they haven't been added yet. Here's an example:

Remove duplicates from a list, while preserving order in Python using a loop

original_list = [1, 2, 3, 4, 2, 5, 3, 6]

# Use a loop to add items to a new list if they haven't been added yet
no_duplicates_list = []
for item in original_list:
    if item not in no_duplicates_list:
        no_duplicates_list.append(item)

print(no_duplicates_list)
# Output: [1, 2, 3, 4, 5, 6]

Note: This approach has O(n²) time complexity because checking item not in no_duplicates_list requires scanning the new list each time. For large datasets, the dict.fromkeys() method is significantly more efficient (O(n)).

Another way is to use collections.OrderedDict() and extract its keys. This is particularly useful for Python versions older than 3.7. Here's an example:

Remove duplicates from a list, while preserving order in Python using OrderedDict

from collections import OrderedDict
original_list = [1, 2, 3, 4, 2, 5, 3, 6]

no_duplicates_list = list(OrderedDict.fromkeys(original_list))

print(no_duplicates_list)
# Output: [1, 2, 3, 4, 5, 6]

You can choose the one which suits your need the best.