Why Python Dictionaries Are Unordered (and How to Keep Insertion Order)

In Python, dictionaries are an essential data structure that stores key-value pairs. As a developer, you might expect a Python dictionary to behave similarly to other data structures, such as C++'s std::map, where elements are sorted by their keys. However, this is not the case. While Python dictionaries have a rich set of features, they do not automatically sort their elements. In this article, we’ll dive into the unordered nature of Python dictionaries, how you can control the order of elements, and how recent changes to Python have impacted this behavior.

Are Python Dictionaries Ordered?

At first glance, Python dictionaries may seem like they should automatically sort the elements by keys. This assumption might arise from the behavior of data structures in other programming languages like C++'s std::map, which maintains elements in sorted order. However, dictionaries in Python do not follow this convention.

Example of Unordered Python Dictionary

Consider the following simple dictionary in Python:

# Regular Python dictionary
myDict = {'first': 1, 'second': 2, 'third': 3}
 
# Printing the dictionary itself
print(myDict)  
# Output: {'first': 1, 'second': 2, 'third': 3}
 
# Iterating through the dictionary keys
print([k for k in myDict])  
# Output: ['second', 'third', 'first']

In the code above:

• When you print myDict, you see the key-value pairs in an order that may seem arbitrary.
• Even though the dictionary was created with a specific order of insertion, when you loop through the keys using list comprehension, the order is different (['second', 'third', 'first']).

This behavior highlights that dictionaries in Python are unordered collections. This means that the order in which elements appear when iterated over is not guaranteed and can vary.

Ordered Dictionaries with OrderedDict

Although the standard Python dictionary does not guarantee the order of keys, Python offers a solution when you need to preserve the order of key-value pairs: OrderedDict. This specialized dictionary, provided by the collections module, maintains the order in which the keys were inserted into the dictionary.

Using OrderedDict to Preserve Insertion Order

Here’s an example of using OrderedDict:

from collections import OrderedDict
 
# Using OrderedDict to preserve insertion order
oDict = OrderedDict([('first', 1), ('second', 2), ('third', 3)])
 
# Printing the OrderedDict and its keys
print([k for k in oDict])  
# Output: ['first', 'second', 'third']

In the code above:

• We import OrderedDict from the collections module.
• By creating an OrderedDict with the same key-value pairs, we ensure that the order of insertion is preserved.
• When iterating through the keys, the output is ['first', 'second', 'third'], exactly as they were inserted.

Key Points about OrderedDict:

1. Preserves Insertion Order: OrderedDict guarantees that elements will be iterated over in the same order they were inserted.
2. No Automatic Sorting: While it preserves order, it does not sort the keys automatically. The order is strictly based on the sequence of insertion.
3. Performance Considerations: OrderedDict may have a slightly higher memory overhead compared to the built-in dict due to the need to preserve order. However, it provides a valuable feature when order is important.

Can You Sort a Dictionary by Keys?

If you need to sort the keys of a dictionary, you cannot rely on the default behavior of Python’s standard dictionary. Instead, you can explicitly sort the dictionary by keys using the sorted() function.

Here’s an example of sorting a dictionary by its keys:

# Standard dictionary
myDict = {'first': 1, 'second': 2, 'third': 3}
 
# Sorting dictionary by keys
sortedDict = {k: myDict[k] for k in sorted(myDict)}
 
print(sortedDict)  
# Output: {'first': 1, 'second': 2, 'third': 3}

In the code above:

• We use the sorted() function to sort the keys of myDict.
• A dictionary comprehension is used to create a new dictionary with the keys in sorted order.

It’s important to note that sorting creates a new dictionary and doesn’t modify the original one.

Python 3.6 and the Change to Dictionary Implementation

Python 3.6 introduced a significant change to the implementation of dictionaries. While dictionaries in Python 2.x and early versions of Python 3 were unordered collections, starting with Python 3.6, the internal implementation of dictionaries was optimized for memory efficiency. A side effect of this optimization is that Python dictionaries now maintain insertion order as an implementation detail.

Example of Python 3.6+ Dictionary Behavior

In Python 3.6 and later versions, dictionaries will maintain the insertion order of keys:

# Python 3.6 and later (with insertion order preservation)
myDict = {'first': 1, 'second': 2, 'third': 3}
 
# Printing the dictionary itself
print(myDict)  
# Output: {'first': 1, 'second': 2, 'third': 3}
 
# Iterating through the dictionary keys
print([k for k in myDict])  
# Output: ['first', 'second', 'third']

While this feature was not explicitly guaranteed in Python 3.5 and earlier, it became a language guarantee starting in Python 3.7. So, if you’re using Python 3.6 or later, dictionaries will remember the order of key insertion, much like OrderedDict. However, it’s still important to understand that this behavior is an implementation detail and should not be relied upon for code that needs compatibility with earlier Python versions.

Python 3.6+ and Keyword Argument Order

An interesting consequence of the change to dictionary implementation in Python 3.6+ is that the order of keyword arguments passed to a function is now preserved. In earlier versions of Python, the order of keyword arguments was not guaranteed, but with Python 3.6 and later, they are passed in the order in which they appear:

def print_args(**kwargs):
    print(kwargs)
 
# Passing keyword arguments
print_args(first=1, second=2, third=3)
# Output: {'first': 1, 'second': 2, 'third': 3}

In this example:

• The order in which the keyword arguments are passed to the function is maintained in the resulting dictionary (kwargs).
• This makes it easier to rely on the order of arguments when working with function signatures, especially in more complex scenarios.

Conclusion: Understanding Python Dictionaries and Their Order

To summarize, Python dictionaries:

• Are unordered collections by default (in Python 2.x and early Python 3.x).
• In Python 3.6 and later, dictionaries preserve the insertion order as an implementation detail, which has become a language guarantee in Python 3.7+.
• Can be sorted manually using the sorted() function, or you can use OrderedDict from the collections module if you explicitly need to preserve insertion order in Python 2.x or earlier versions of Python 3.

By understanding the nuances of dictionary ordering in Python, you can avoid potential pitfalls in your code and ensure that your program behaves as expected when working with key-value pairs.

Call to Action

If you’re working with legacy Python code that depends on unordered dictionaries or requires sorting of dictionary keys, consider upgrading to Python 3.7+ to take advantage of the built-in insertion order preservation. If you need more control over the ordering behavior, OrderedDict remains a powerful tool for those scenarios. For more tips on Python dictionaries, check out the official Python documentation or join discussions in Python communities to stay updated on best practices!