Creating calculated columns from existing data is one of the most fundamental operations in data transformation. Whether you are computing totals from prices and quantities, deriving new metrics from raw measurements, or building features for a machine learning model, Pandas provides vectorized operations that process entire columns at once. This makes calculations fast, concise, and easy to read.
In this guide, you will learn how to add calculated columns using direct assignment and the .assign() method, understand why vectorized operations vastly outperform loops, and explore common calculation patterns you will encounter in everyday data work.
Direct Assignment: In-Place Column Creation
The simplest way to add a calculated column is to assign an expression directly to a new column name. Pandas evaluates the expression across all rows simultaneously:
import pandas as pd
df = pd.DataFrame({
'Price': [10, 20, 15],
'Qty': [5, 3, 4]
})
df['Total'] = df['Price'] * df['Qty']
print(df)Output:
Price Qty Total
0 10 5 50
1 20 3 60
2 15 4 60Building Multiple Calculated Columns
You can chain multiple assignments, with each new column referencing previously created ones:
import pandas as pd
df = pd.DataFrame({
'Price': [100, 200, 150],
'Qty': [2, 3, 4]
})
df['Subtotal'] = df['Price'] * df['Qty']
df['Tax'] = df['Subtotal'] * 0.08
df['Total'] = df['Subtotal'] + df['Tax']
print(df)Output:
Price Qty Subtotal Tax Total
0 100 2 200 16.0 216.0
1 200 3 600 48.0 648.0
2 150 4 600 48.0 648.0Using .assign(): Returning a New DataFrame
The .assign() method creates and returns a new DataFrame with the added columns, leaving the original untouched. This makes it ideal for method chaining and functional-style workflows:
import pandas as pd
df = pd.DataFrame({
'Price': [10, 20],
'Qty': [5, 3]
})
df_new = df.assign(
Total=lambda x: x['Price'] * x['Qty'],
Tax=lambda x: x['Price'] * 0.1
)
print(df_new)
print()
print(df)Output:
Price Qty Total Tax
0 10 5 50 1.0
1 20 3 60 2.0
Price Qty
0 10 5
1 20 3Chaining Operations with .assign()
The real strength of .assign() is building readable transformation pipelines:
import pandas as pd
df = pd.DataFrame({
'Price': [10, 20, 15],
'Qty': [5, 3, 4]
})
result = (df
.assign(Subtotal=lambda x: x['Price'] * x['Qty'])
.assign(Tax=lambda x: x['Subtotal'] * 0.08)
.assign(Total=lambda x: x['Subtotal'] + x['Tax'])
.query('Total > 50')
)
print(result)Output:
Price Qty Subtotal Tax Total
0 10 5 50 4.0 54.0
1 20 3 60 4.8 64.8
2 15 4 60 4.8 64.8Why You Should Avoid Loops
A common mistake, especially for those coming from other programming languages, is using row-by-row iteration to compute new columns. This approach is dramatically slower than vectorized operations because it bypasses Pandas' optimized C-based internals.
import pandas as pd
df = pd.DataFrame({'Price': [10, 20], 'Qty': [5, 3]})
for i, row in df.iterrows():
df.loc[i, 'Total'] = row['Price'] * row['Qty']
print(df)Output:
Price Qty Total
0 10 5 50.0
1 20 3 60.0The vectorized equivalent produces the same result but runs orders of magnitude faster:
import pandas as pd
df = pd.DataFrame({'Price': [10, 20], 'Qty': [5, 3]})
df['Total'] = df['Price'] * df['Qty']
print(df)Output:
Price Qty Total
0 10 5 50
1 20 3 60Performance Difference
To illustrate the scale of the difference, here is a timing comparison on a DataFrame with 10,000 rows:
import pandas as pd
import time
df = pd.DataFrame({'A': range(10000), 'B': range(10000)})
start = time.time()
df['C'] = df['A'] * df['B']
vectorized_time = time.time() - start
start = time.time()
for i, row in df.iterrows():
df.loc[i, 'C'] = row['A'] * row['B']
loop_time = time.time() - start
print(f"Vectorized: {vectorized_time:.4f}s")
print(f"Loop: {loop_time:.4f}s")
print(f"Loop is ~{loop_time / vectorized_time:.0f}x slower")Output:
Vectorized: 0.0017s
Loop: 1.5670s
Loop is ~932x slowerCommon Calculation Patterns
Arithmetic Operations
import pandas as pd
df = pd.DataFrame({
'A': [10, 20, 30],
'B': [3, 4, 5]
})
df['Sum'] = df['A'] + df['B']
df['Diff'] = df['A'] - df['B']
df['Product'] = df['A'] * df['B']
df['Ratio'] = df['A'] / df['B']
print(df)Output:
A B Sum Diff Product Ratio
0 10 3 13 7 30 3.333333
1 20 4 24 16 80 5.000000
2 30 5 35 25 150 6.000000Mathematical Functions with NumPy
import pandas as pd
import numpy as np
df = pd.DataFrame({
'A': [10, 20, 30]
})
df['Log_A'] = np.log(df['A'])
df['Sqrt_A'] = np.sqrt(df['A'])
print(df)Output:
A Log_A Sqrt_A
0 10 2.302585 3.162278
1 20 2.995732 4.472136
2 30 3.401197 5.477226String Concatenation
import pandas as pd
df = pd.DataFrame({
'First': ['John', 'Jane'],
'Last': ['Doe', 'Smith']
})
df['Full_Name'] = df['First'] + ' ' + df['Last']
print(df)Output:
First Last Full_Name
0 John Doe John Doe
1 Jane Smith Jane SmithConditional Columns
Use np.where() for simple conditions or .apply() for more complex logic:
import pandas as pd
import numpy as np
df = pd.DataFrame({'Score': [85, 45, 72, 90, 55]})
df['Pass'] = np.where(df['Score'] >= 60, 'Yes', 'No')
df['Grade'] = df['Score'].apply(
lambda x: 'A' if x >= 90 else ('B' if x >= 80 else ('C' if x >= 60 else 'F'))
)
print(df)Output:
Score Pass Grade
0 85 Yes B
1 45 No F
2 72 Yes C
3 90 Yes A
4 55 No FQuick Reference
| Method | Modifies Original | Best For |
|---|---|---|
df['col'] = expr |
Yes | Simple scripts, quick modifications |
df.assign(col=expr) |
No | Method chaining, preserving the original |
| Operation Type | Example |
|---|---|
| Arithmetic | df['C'] = df['A'] + df['B'] |
| Math function | df['Log'] = np.log(df['A']) |
| Conditional | df['Flag'] = np.where(df['A'] > 0, 1, 0) |
| String | df['Full'] = df['First'] + ' ' + df['Last'] |
- Use direct assignment (
df['col'] = expression) for straightforward, in-place calculations. - Use
.assign()when you need method chaining or want to keep the original DataFrame unchanged.
In all cases, prefer vectorized operations over loops for performance gains that can reach several hundred times faster on real-world datasets.