Skip to content
Tutorial Reference

How to Flatten a List of DataFrames in Python

Learn how to combine multiple DataFrames into one using pd.concat() and flatten nested list columns into rows with explode() for efficient data analysis in pandas.

When working with pandas in Python, you frequently encounter scenarios where data is stored as a list of DataFrames - for example, when reading multiple CSV files, splitting data by groups, or collecting results from batch operations. Flattening these into a single DataFrame makes the data easier to analyze, query, and export.

This guide covers how to flatten (combine) a list of separate DataFrames into one, as well as how to flatten list-like values stored within DataFrame columns.

Combining a List of DataFrames into One

The most common "flattening" task is merging multiple DataFrames into a single DataFrame. The pd.concat() function is the standard and most efficient way to do this.

import pandas as pd

df1 = pd.DataFrame({"name": ["Alice", "Bob"], "score": [85, 92]})
df2 = pd.DataFrame({"name": ["Charlie", "Diana"], "score": [78, 95]})
df3 = pd.DataFrame({"name": ["Eve"], "score": [88]})

list_of_dfs = [df1, df2, df3]

combined = pd.concat(list_of_dfs, ignore_index=True)
print(combined)

Output:

      name  score
0    Alice     85
1      Bob     92
2  Charlie     78
3    Diana     95
4      Eve     88
  • pd.concat(list_of_dfs) stacks all DataFrames vertically (row-wise) by default.
  • ignore_index=True resets the index to a continuous sequence (0, 1, 2, ...) instead of preserving original indices.

Handling DataFrames with Different Columns

If the DataFrames have different columns, pd.concat() aligns them and fills missing values with NaN:

import pandas as pd

df1 = pd.DataFrame({"name": ["Alice"], "score": [85]})
df2 = pd.DataFrame({"name": ["Bob"], "grade": ["A"]})

combined = pd.concat([df1, df2], ignore_index=True)
print(combined)

Output:

    name  score grade
0  Alice   85.0   NaN
1    Bob    NaN     A

Flattening List Values Inside DataFrame Columns

A different type of flattening involves expanding list-like values stored within a column so that each element gets its own row.

The explode() method (available in pandas 0.25+) is the simplest way to flatten a column containing lists:

import pandas as pd

df = pd.DataFrame({
    "payments": [[300, 400, 500, 600], [300, 322, 333, 233]],
    "name": ["sravan", "bobby"]
})

print("Original DataFrame:")
print(df)
print()

flattened = df.explode("payments", ignore_index=True)

print("Flattened DataFrame:")
print(flattened)

Output:

Original DataFrame:
               payments    name
0  [300, 400, 500, 600]  sravan
1  [300, 322, 333, 233]   bobby

Flattened DataFrame:
  payments    name
0      300  sravan
1      400  sravan
2      500  sravan
3      600  sravan
4      300   bobby
5      322   bobby
6      333   bobby
7      233   bobby

Each element in the payments list gets its own row, and the corresponding name value is repeated for each entry.

Exploding Multiple Columns

Starting with pandas 1.3.0, you can explode multiple columns simultaneously (they must have lists of the same length in each row):

import pandas as pd

df = pd.DataFrame({
    "months": [["Jan", "Feb", "Mar"], ["Jan", "Feb", "Mar"]],
    "payments": [[300, 400, 500], [100, 200, 150]],
    "name": ["Alice", "Bob"]
})

flattened = df.explode(["months", "payments"], ignore_index=True)
print(flattened)

Output:

  months payments   name
0    Jan      300  Alice
1    Feb      400  Alice
2    Mar      500  Alice
3    Jan      100    Bob
4    Feb      200    Bob
5    Mar      150    Bob

Manual Approach (Pre-pandas 0.25)

If you're working with an older version of pandas that doesn't have explode(), you can flatten list columns manually using iteration:

import pandas as pd

df = pd.DataFrame({
    "payments": [[300, 400, 500, 600], [300, 322, 333, 233]],
    "name": ["sravan", "bobby"]
})

flat_data = pd.DataFrame(
    [(idx, value) for idx, values in df["payments"].items() for value in values],
    columns=["index", "payments"]
).set_index("index")

result = df.drop("payments", axis=1).join(flat_data)
print(result)

Output:

     name  payments
0  sravan       300
0  sravan       400
0  sravan       500
0  sravan       600
1   bobby       300
1   bobby       322
1   bobby       333
1   bobby       233

Flattening a DataFrame's Values to a 1D Array

If you need to flatten an entire DataFrame into a one-dimensional array (all values in a single flat sequence), use NumPy's .flatten() on the underlying values:

import pandas as pd

df = pd.DataFrame({
    "A": [1, 2, 3],
    "B": [4, 5, 6]
})

flat_array = df.values.flatten()
print(flat_array)
print(type(flat_array))

Output:

[1 4 2 5 3 6]
<class 'numpy.ndarray'>

Practical Example: Reading and Combining Multiple CSV Files

A real-world use case for flattening a list of DataFrames is loading multiple files into a single dataset:

import pandas as pd
from pathlib import Path

data_dir = Path("data/reports/")
csv_files = list(data_dir.glob("*.csv"))

dfs = [pd.read_csv(file) for file in csv_files]

combined = pd.concat(dfs, ignore_index=True)

print(f"Loaded {len(csv_files)} files with {len(combined)} total rows")
print(combined.head())

Quick Reference

Task Method Best For
Combine list of DataFrames pd.concat(dfs, ignore_index=True) Merging multiple DataFrames
Flatten list column to rows df.explode("column") Expanding list values (pandas 0.25+)
Flatten multiple list columns df.explode(["col1", "col2"]) Parallel list expansion (pandas 1.3+)
Flatten all values to 1D array df.values.flatten() Converting entire DataFrame to array

Conclusion

Flattening DataFrames in Python typically falls into two categories:

  • combining multiple DataFrames into one using pd.concat(),
  • or expanding list-like values within columns using explode().

Both operations are essential for data preparation and analysis. Use pd.concat() with ignore_index=True for clean merging, and explode() for transforming nested list data into a flat, row-per-value format that's ready for analysis.

Navigation