Skip to content
Tutorial Reference

How to Aggregate Lists of Dictionaries in Python

Learn efficient Python techniques for summing values, grouping data, and calculating statistics from lists of dictionaries using built-in functions, itertools, and pandas.

Aggregating data, summarizing, grouping, or transforming, is a daily task for Python developers. When working with lists of dictionaries (a common format for JSON data or database records), knowing the right tools for aggregation is crucial for performance and readability.

This guide demonstrates efficient methods to sum values, group items, and calculate statistics from lists of dictionaries.

Sample Data

We will use a list of sales records for our examples.

sales_data = [
    {"product": "Laptop", "category": "Electronics", "price": 1000, "quantity": 2},
    {"product": "Phone",  "category": "Electronics", "price": 500,  "quantity": 5},
    {"product": "Book",   "category": "Books",       "price": 20,   "quantity": 10},
    {"product": "Tablet", "category": "Electronics", "price": 300,  "quantity": 3},
    {"product": "Pen",    "category": "Stationery",  "price": 5,    "quantity": 50}
]

Method 1: Using sum() and Generator Expressions

For simple calculations like total revenue or total items sold, Python's built-in sum() combined with a generator expression is the most pythonic and memory-efficient way.

sales_data = [
    {"product": "Laptop", "category": "Electronics", "price": 1000, "quantity": 2},
    {"product": "Phone",  "category": "Electronics", "price": 500,  "quantity": 5},
    {"product": "Book",   "category": "Books",       "price": 20,   "quantity": 10},
    {"product": "Tablet", "category": "Electronics", "price": 300,  "quantity": 3},
    {"product": "Pen",    "category": "Stationery",  "price": 5,    "quantity": 50}
]

total_revenue = sum(item['price'] * item['quantity'] for item in sales_data)

total_quantity = sum(item['quantity'] for item in sales_data)

print(f"Total Revenue: ${total_revenue}")
print(f"Total Items Sold: {total_quantity}")

Output:

Total Revenue: $5850
Total Items Sold: 70

Method 2: Grouping with collections.defaultdict

If you need to group data (e.g., total sales per category), defaultdict is often cleaner than using standard dictionary checks.

from collections import defaultdict

sales_data = [
    {"product": "Laptop", "category": "Electronics", "price": 1000, "quantity": 2},
    {"product": "Phone",  "category": "Electronics", "price": 500,  "quantity": 5},
    {"product": "Book",   "category": "Books",       "price": 20,   "quantity": 10},
    {"product": "Tablet", "category": "Electronics", "price": 300,  "quantity": 3},
    {"product": "Pen",    "category": "Stationery",  "price": 5,    "quantity": 50}
]

category_sales = defaultdict(int)

for item in sales_data:
    category = item['category']
    revenue = item['price'] * item['quantity']
    
    category_sales[category] += revenue

print("Revenue by Category:")
for category, amount in category_sales.items():
    print(f"{category}: ${amount}")

Output:

Revenue by Category:
Electronics: $5400
Books: $200
Stationery: $250

Method 3: Grouping with itertools.groupby

itertools.groupby is powerful but requires the input list to be sorted by the grouping key first. It is useful when you need to iterate through groups rather than just sum them.

from itertools import groupby

sales_data = [
    {"product": "Laptop", "category": "Electronics", "price": 1000, "quantity": 2},
    {"product": "Phone",  "category": "Electronics", "price": 500,  "quantity": 5},
    {"product": "Book",   "category": "Books",       "price": 20,   "quantity": 10},
    {"product": "Tablet", "category": "Electronics", "price": 300,  "quantity": 3},
    {"product": "Pen",    "category": "Stationery",  "price": 5,    "quantity": 50}
]

sorted_sales = sorted(sales_data, key=lambda x: x['category'])

print("Items in each category:")
for category, group in groupby(sorted_sales, key=lambda x: x['category']):
    items = list(group)
    item_names = [item['product'] for item in items]
    print(f"{category}: {item_names}")

Output:

Items in each category:
Books: ['Book']
Electronics: ['Laptop', 'Phone', 'Tablet']
Stationery: ['Pen']

Method 4: Using Pandas (For Large Datasets)

For complex aggregations or large datasets, pandas is the industry standard. It handles grouping and aggregation with SQL-like syntax efficiently.

import pandas as pd

sales_data = [
    {"product": "Laptop", "category": "Electronics", "price": 1000, "quantity": 2},
    {"product": "Phone",  "category": "Electronics", "price": 500,  "quantity": 5},
    {"product": "Book",   "category": "Books",       "price": 20,   "quantity": 10},
    {"product": "Tablet", "category": "Electronics", "price": 300,  "quantity": 3},
    {"product": "Pen",    "category": "Stationery",  "price": 5,    "quantity": 50}
]

df = pd.DataFrame(sales_data)

df['revenue'] = df['price'] * df['quantity']

category_summary = df.groupby('category')['revenue'].sum()

print(category_summary)

Output:

category
Books           200
Electronics    5400
Stationery      250
Name: revenue, dtype: int64

Conclusion

Choosing the right aggregation method depends on your needs:

  1. Simple Sums: Use sum() with generator expressions.
  2. Grouping & Summing: Use collections.defaultdict.
  3. Grouping & Iterating: Use itertools.groupby (remember to sort first!).
  4. Complex Analysis: Use pandas for powerful DataFrames.
Navigation