Skip to content
Tutorial Reference

Python Pandas: How to Do Element-wise Logical NOT ('~') and OR ('|') Operations

Learn how to perform element-wise logical NOT (~) and OR (|) operations in Pandas on boolean Series for complex data filtering and condition-based row selection.

Performing element-wise logical operations on boolean Series (often derived from conditions on DataFrame columns) is fundamental for complex data filtering and manipulation in Pandas. The logical NOT operation inverts boolean values, while the logical OR operation combines conditions, selecting rows if at least one condition is true.

This guide explains how to use the tilde (~) operator for logical NOT and the pipe (|) operator for logical OR in Pandas, as well as their NumPy equivalents.

The Goal: Boolean Logic on Series/DataFrame

  • Logical NOT: Given a boolean Series (e.g., df['Column'] > 10), we want to invert its values (True becomes False, False becomes True).
  • Logical OR: Given two or more boolean Series (conditions), we want to create a new boolean Series that is True if at least one of the input conditions is True for a given row.

This is essential for building complex filters for DataFrame row selection.

Example DataFrame

import pandas as pd
import numpy as np # For NaN and numpy functions

data = {
    'EmployeeID': ['E01', 'E02', 'E03', 'E04', 'E05'],
    'Department': ['Sales', 'HR', 'Engineering', 'Sales', 'HR'],
    'YearsExperience': [2, 5, 1, 7, 4],
    'IsManager': [False, True, False, True, False],
    'OnProject': [True, False, True, True, np.nan] # Contains NaN
}

df = pd.DataFrame(data)
print("Original DataFrame:")
print(df)

Output:

Original DataFrame:
  EmployeeID   Department  YearsExperience  IsManager OnProject
0        E01        Sales                2      False      True
1        E02           HR                5       True     False
2        E03  Engineering                1      False      True
3        E04        Sales                7       True      True
4        E05           HR                4      False       NaN

Element-wise Logical NOT

Using the Tilde Operator (~) (Recommended)

The tilde ~ is the unary operator for bitwise NOT in Python, and Pandas overloads it for element-wise logical NOT on boolean Series/DataFrames.

import pandas as pd

df_example = pd.DataFrame({
    'IsManager': [False, True, False, True, False]
})

is_manager_series = df_example['IsManager']
print("Original 'IsManager' Series:")
print(is_manager_series)
print()

is_not_manager_series = ~is_manager_series
print("Inverted 'IsManager' Series (is_not_manager_series):")
print(is_not_manager_series)

Output:

Original 'IsManager' Series:
0    False
1     True
2    False
3     True
4    False
Name: IsManager, dtype: bool

Inverted 'IsManager' Series (is_not_manager_series):
0     True
1    False
2     True
3    False
4     True
Name: IsManager, dtype: bool

Using the Unary Minus Operator (-) (Less Common for Booleans)

Pandas also allows the unary minus - to act as a logical NOT for boolean Series. This is less conventional for boolean logic compared to ~.

import pandas as pd

df_example = pd.DataFrame({
    'IsManager': [False, True, False, True, False]
})

is_manager_series = df_example['IsManager']

is_not_manager_minus = -is_manager_series
print("Inverted using unary minus (-):")
print(is_not_manager_minus) # Output is same as with ~

Output:

Inverted using unary minus (-):
0     True
1    False
2     True
3    False
4     True
Name: IsManager, dtype: bool

Using numpy.invert()

NumPy's np.invert() function performs a bitwise NOT, which also works for boolean arrays/Series.

import pandas as pd
import numpy as np

df_example = pd.DataFrame({ 'IsManager': [False, True, False, True, False] })
is_manager_series = df_example['IsManager']

is_not_manager_np = np.invert(is_manager_series)
print("Inverted 'IsManager' Series (using np.invert()):")
print(is_not_manager_np)

Output:

Inverted 'IsManager' Series (using np.invert()):
0     True
1    False
2     True
3    False
4     True
Name: IsManager, dtype: bool

Handling Missing Values (NaN) Before NOT

The ~ operator (and np.invert) on a boolean Series containing NaN will treat NaN as True (because NaN is falsy, so not NaN is True in some boolean contexts, or more precisely, non-True values become True). If you need NaNs to remain NaN or be treated as False after inversion, you might need to handle them first or use astype(bool) carefully.

import pandas as pd
import numpy as np

s_with_nan = pd.Series([True, False, np.nan, True])
print(f"Original Series with NaN:\n{s_with_nan}\n")

s_nullable_bool = s_with_nan.astype("boolean")

print(f"~s_with_nan (nullable boolean):\n{~s_nullable_bool}\n")

s_astype_bool = s_with_nan.fillna(True).astype(bool)
print(f"s_with_nan.astype(bool) [NaN -> True]:\n{s_astype_bool}\n")   # NaN becomes True
print(f"~s_with_nan.astype(bool):\n{~s_astype_bool}")  # Then invert

Output:

Original Series with NaN:
0     True
1    False
2      NaN
3     True
dtype: object

~s_with_nan (nullable boolean):
0    False
1     True
2     <NA>
3    False
dtype: boolean

s_with_nan.astype(bool) [NaN -> True]:
0     True
1    False
2     True
3     True
dtype: bool

~s_with_nan.astype(bool):
0    False
1     True
2    False
3    False
dtype: bool

Element-wise Logical OR

Using the Pipe Operator (|) (Recommended)

The pipe | is the bitwise OR operator in Python. Pandas overloads it for element-wise logical OR between boolean Series.

import pandas as pd

df_example = pd.DataFrame({
    'Department': ['Sales', 'HR', 'Engineering', 'Sales'],
    'YearsExperience': [2, 5, 1, 7],
    'IsManager': [False, True, False, True]
})

cond1 = (df_example['Department'] == 'HR')
cond2 = (df_example['IsManager'] == True)

combined_or_mask = cond1 | cond2
print("Combined OR mask (HR or IsManager):")
print(combined_or_mask)
print()

print("Rows where Department is 'HR' OR IsManager is True:")
print(df_example[combined_or_mask])

Output:

Combined OR mask (HR or IsManager):
0    False
1     True
2    False
3     True
dtype: bool

Rows where Department is 'HR' OR IsManager is True:
  Department  YearsExperience  IsManager
1         HR                5       True
3      Sales                7       True

Using numpy.logical_or()

NumPy's np.logical_or(condition1, condition2) performs an element-wise logical OR.

import pandas as pd
import numpy as np

df_example = pd.DataFrame({
    'Department': ['Sales', 'HR'], 'IsManager': [False, True]
})
cond1 = (df_example['Department'] == 'HR')
cond2 = (df_example['IsManager'] == True)

combined_or_np = np.logical_or(cond1, cond2)
print("Combined OR mask (using np.logical_or()):")
print(combined_or_np) # Returns a NumPy array
print()

print("Rows selected using np.logical_or():")
print(df_example[combined_or_np])

Output:

Combined OR mask (using np.logical_or()):
0    False
1     True
dtype: bool

Rows selected using np.logical_or():
  Department  IsManager
1         HR       True

Using numpy.logical_or.reduce() for Multiple Conditions

If you have a list of more than two boolean Series (conditions) and want to OR them all together, np.logical_or.reduce() is useful.

import pandas as pd
import numpy as np

df_example = pd.DataFrame({
    'Department': ['Sales', 'HR', 'Dev', 'Sales'],
    'YearsExperience': [2, 5, 1, 7],
    'IsManager': [False, True, False, False]
})

cond_dept_hr = (df_example['Department'] == 'HR')
cond_is_manager = (df_example['IsManager'] == True)
cond_exp_gt_5 = (df_example['YearsExperience'] > 5)

list_of_conditions = [cond_dept_hr, cond_is_manager, cond_exp_gt_5]

combined_or_reduce = np.logical_or.reduce(list_of_conditions)
print("Combined OR mask (using np.logical_or.reduce()):")
print(combined_or_reduce)
print()

print("Rows selected using np.logical_or.reduce():")
print(df_example[combined_or_reduce])

Output:

Combined OR mask (using np.logical_or.reduce()):
[False  True False  True]

Rows selected using np.logical_or.reduce():
  Department  YearsExperience  IsManager
1         HR                5       True
3      Sales                7      False

Operator Precedence and Parentheses

When combining multiple logical conditions (especially with & AND and | OR), it's crucial to use parentheses () to ensure correct order of operations due to Python's operator precedence.

  • & (AND) has higher precedence than | (OR).
  • ~ (NOT) has higher precedence than & and |.

Always wrap individual conditions in parentheses when combining them:

mask = (df['colA'] > 10) & (df['colB'] == 'X') | (df['colC'] < 0)

mask = df['colA'] > 10 & df['colB'] == 'X' | df['colC'] < 0

Conclusion

Pandas leverages Python's bitwise operators for element-wise logical operations on boolean Series:

  • For Logical NOT (inversion): Use the tilde operator (~) directly on a boolean Series: ~your_boolean_series.
  • For Logical OR (select if any condition is true): Use the pipe operator (|) between boolean Series: (condition1) | (condition2). Remember to use parentheses around individual conditions.
  • NumPy functions like np.invert(), np.logical_or(), and np.logical_or.reduce() provide alternatives, though the Pandas operators (~, |) are often more concise and idiomatic for DataFrame operations.
  • Be mindful of how NaN values are treated, especially with the NOT operator, and consider using astype(bool) or fillna() if specific handling of missing boolean values is required before inversion.

These logical operators are fundamental for constructing complex boolean masks used in filtering and selecting data within your Pandas DataFrames.

Navigation