How to Get Unique Values from a List in Python

Extracting unique values from a list, removing duplicates while keeping only distinct elements, is one of the most common operations in Python. Whether you're cleaning data, processing user inputs, or preparing datasets, knowing how to efficiently deduplicate a list is essential.

For example, given [1, 2, 1, 1, 3, 4, 3, 3, 5], the unique values are [1, 2, 3, 4, 5].

This guide covers several approaches, highlighting the trade-off between preserving order and performance.

Using dict.fromkeys() (Recommended)

The most Pythonic and efficient way to get unique values while preserving the original order is dict.fromkeys(). Since dictionary keys must be unique (and since Python 3.7+, dictionaries maintain insertion order), this approach removes duplicates and keeps the first occurrence of each element.

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

unique = list(dict.fromkeys(a))
print(unique)

Output:

[1, 2, 3, 4, 5]

dict.fromkeys(a) creates a dictionary where each element from the list becomes a key. Duplicate elements are naturally discarded since keys can't repeat. Converting the keys back to a list produces the unique values in their original order.

Using set() (Fastest, No Order Guarantee)

Converting a list to a set is the fastest way to remove duplicates because sets are optimized for uniqueness checks. However, sets are unordered, so the original element order is not preserved.

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

unique = list(set(a))
print(unique)

Output (order may vary):

[1, 2, 3, 4, 5]

Preserving Order with set + sorted()

If you need sorted unique values (not original order, but sorted), combine set with sorted():

a = [5, 3, 1, 4, 2, 3, 1, 5]

unique_sorted = sorted(set(a))
print(unique_sorted)

Output:

[1, 2, 3, 4, 5]

Using collections.OrderedDict

Before Python 3.7 guaranteed dictionary insertion order, OrderedDict was the standard way to remove duplicates while preserving order. It still works perfectly and makes the intent explicit:

from collections import OrderedDict

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

unique = list(OrderedDict.fromkeys(a))
print(unique)

Output:

[1, 2, 3, 4, 5]

Using a for Loop

A manual loop gives you the most control and is easy to understand. It checks each element before adding it to the result list:

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

unique = []
for item in a:
    if item not in unique:
        unique.append(item)

print(unique)

Output:

[1, 2, 3, 4, 5]

This preserves order and is clear in its intent, but it has a performance drawback.

Using List Comprehension with a Set

A concise version of the optimized loop approach using list comprehension:

a = [1, 2, 1, 1, 3, 4, 3, 3, 5]

seen = set()
unique = [x for x in a if x not in seen and not seen.add(x)]
print(unique)

Output:

[1, 2, 3, 4, 5]

This works because seen.add(x) returns None (which is falsy), so not seen.add(x) is always True. The side effect of adding x to seen happens during the comprehension.

Working with Unhashable Elements

All the methods above require elements to be hashable (e.g., numbers, strings, tuples). If your list contains unhashable elements like dictionaries or lists, you need a different approach:

a = [{'name': 'Alice'}, {'name': 'Bob'}, {'name': 'Alice'}]

unique = []
for item in a:
    if item not in unique:
        unique.append(item)

print(unique)

Output:

[{'name': 'Alice'}, {'name': 'Bob'}]

This O(n²) loop is the only option for unhashable types since they cannot be added to sets or used as dictionary keys.

Comparison of Approaches

Conclusion

For getting unique values from a list in Python, dict.fromkeys() is the best default choice: it's fast, preserves order, and requires no imports.

• Use set() when you do not care about order and want maximum speed.
• For large lists where you need both order and performance with custom logic
• Use a loop with a set for tracking.
• When working with unhashable elements like dictionaries or nested lists, a simple for loop with membership checking is your only option.