How to Run Tasks Concurrently in Python Using Threads

Python's threading module allows you to run multiple operations concurrently, making your programs more efficient, especially when dealing with I/O-bound tasks (like reading files or network requests). This guide covers creating threads, synchronizing them, and using advanced tools like ThreadPoolExecutor and synchronization primitives.

Basic Threading

The simplest way to run code in parallel is to create a Thread object, passing it a target function.

import threading
import time

def worker(name):
    print(f"Thread {name} starting...")
    time.sleep(1)
    print(f"Thread {name} finished.")

t = threading.Thread(target=worker, args=("A",))
t.start()

t.join()
print("Done.")

Synchronization with Locks

When multiple threads modify the same variable (a "race condition"), data corruption can occur. A Lock ensures only one thread accesses a resource at a time.

import threading

counter = 0
lock = threading.Lock()

def increment():
    global counter
    with lock:  # ✅ Correct: Automatically acquires and releases
        counter += 1

threads = [threading.Thread(target=increment) for _ in range(100)]

for t in threads: t.start()
for t in threads: t.join()

print(f"Final counter: {counter}")

Thread Pools (Best Practice)

Managing threads manually (starting, joining, storing in lists) is tedious. The concurrent.futures.ThreadPoolExecutor handles the lifecycle of threads automatically.

Using executor.submit()

import concurrent.futures

def task(n):
    return n * n

with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
    future = executor.submit(task, 5)
    print(f"Result: {future.result()}")

Using executor.map()

with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
    items = [1, 2, 3, 4, 5]
    results = executor.map(task, items)
    
    for res in results:
        print(res)

Synchronization Primitives (Event, Timer, Barrier)

Python provides specialized objects to coordinate threads.

Event

Allows one thread to signal others to proceed. Ideally used when threads need to wait for a specific condition (e.g., "Database is ready").

ready_event = threading.Event()

def waiter():
    print("Waiting for signal...")
    ready_event.wait()  # Blocks here
    print("Signal received!")

t = threading.Thread(target=waiter)
t.start()

ready_event.set()

Timer

Runs a function after a delay in a separate thread.

def delayed_action():
    print("Timer finished!")

t = threading.Timer(2.0, delayed_action)
t.start()

Barrier

Forces a group of threads to wait until all of them reach a certain point.

barrier = threading.Barrier(3)

def worker():
    print("Worker ready")
    barrier.wait() # Blocks until 3 threads arrive here
    print("Worker GO!")

for _ in range(3):
    threading.Thread(target=worker).start()

Conclusion

To effectively use threading in Python:

1. Use ThreadPoolExecutor for most tasks; it's cleaner and safer than manual thread management.
2. Use Lock whenever threads modify shared state.
3. Use Event or Barrier for complex coordination between threads.
4. Use Daemon Threads (daemon=True) for background tasks that shouldn't prevent the program from exiting.