How to Configure Levels and Outputs of Logging Module in Python

Proper logging is the backbone of monitoring and debugging Python applications. Unlike print() statements, the built-in logging module allows you to categorize messages by severity (Levels) and route them to different destinations (Outputs), such as the console, files, or external monitoring services.

This guide explains how to control verbosity using logging levels and how to configure handlers to direct logs to specific outputs.

Understanding Logging Levels

The Python logging system uses a hierarchy of levels. When you set a logging level, you are telling the application to record events of that severity and higher.

Method 1: Basic Configuration (Console Output)

By default, Python logs to the console (stderr). However, the default level is WARNING, meaning DEBUG and INFO messages are ignored unless configured otherwise.

Setting the Threshold

Use logging.basicConfig() to set the minimum reporting level.

import logging


logging.basicConfig(level=logging.INFO)

logging.debug("This is a debug message (Hidden, because DEBUG < INFO)")
logging.info("System startup complete.")
logging.warning("Disk space low.")
logging.error("Database connection failed.")

Output:

INFO:root:System startup complete.
WARNING:root:Disk space low.
ERROR:root:Database connection failed.

Method 2: Writing Logs to a File

To persist logs for later analysis, you can redirect output to a file using the filename parameter.

import logging

logging.basicConfig(
    filename='app.log', 
    filemode='w', 
    level=logging.DEBUG
)

logging.info("This message is written to the file, not the console.")
logging.error("An error occurred.")

Content of app.log:

INFO:root:This message is written to the file, not the console.
ERROR:root:An error occurred.

Method 3: Using Multiple Outputs (Handlers)

In production applications, you often want INFO logs to go to the console and ERROR logs (or everything) to be saved to a file. To achieve this, you must attach specific Handlers to the logger.

Configuring Console and File Simultaneously

You can define a list of handlers inside basicConfig.

import logging
import sys

file_handler = logging.FileHandler('server.log')

console_handler = logging.StreamHandler(sys.stdout)

logging.basicConfig(
    level=logging.DEBUG, # Global minimum level
    format='%(asctime)s - %(levelname)s - %(message)s',
    handlers=[file_handler, console_handler]
)

logging.info("Server started on port 8080")
logging.warning("High memory usage detected")

Output (Console):

2023-10-27 10:00:01,123 - INFO - Server started on port 8080
2023-10-27 10:00:01,124 - WARNING - High memory usage detected

Conclusion

To manage logs effectively in Python:

1. Use logging.basicConfig(level=...) to control verbosity.
2. Use filename='...' to save logs for historical analysis.
3. Use Handlers (FileHandler, StreamHandler) when you need to send logs to multiple destinations simultaneously or apply different formatting rules.