How to Unzip .gz Files in Python

This guide explains how to unzip .gz files (GZIP compressed files) in Python. We'll cover both extracting the contents to a new file and reading the uncompressed data directly into memory, using the built-in gzip and shutil modules.

Unzipping a .gz File to a New File (Extraction)

To unzip a .gz file and save the uncompressed contents to a new file, use the gzip and shutil modules:

import gzip
import shutil

with gzip.open('example.json.gz', 'rb') as file_in:
    with open('example.json', 'wb') as file_out:  # wb = write bytes
        shutil.copyfileobj(file_in, file_out)
        print('example.json file created')

• gzip.open('example.json.gz', 'rb'): Opens the .gz file in binary read mode ('rb'). gzip.open() handles the decompression.
• with open('example.json', 'wb') as file_out:: Opens the output file in binary write mode ('wb'). It's crucial to use binary mode here, even if the uncompressed content is text, because the output of gzip.open is bytes.
• shutil.copyfileobj(file_in, file_out): This efficiently copies the uncompressed data from the input file object (file_in) to the output file object (file_out). It handles reading and writing in chunks, so it works well even with large files.
• The code assumes there is a file named example.json.gz in the same directory as the Python script, but you can use any other path instead.

Reading the Uncompressed Contents of a .gz File

If you just want to read the uncompressed data into a Python variable (without creating a new file), you can use gzip.open() and read():

import gzip

with gzip.open('example.json.gz', 'rb') as file_in:
    file_contents = file_in.read()      # Reads as bytes
    print(file_contents)                # Output a bytes object: b'...'

with gzip.open('example.json.gz', 'rt', encoding='utf-8') as file_in:
    file_contents = file_in.read()      # Read as a string directly
    print(file_contents)

• file_in.read(): Reads the entire uncompressed content into the file_contents variable. If it is a text file, the rb should be replaced with rt, and the encoding should be provided.

For line-by-line reading:

import gzip

with gzip.open('example.json.gz', 'rt', encoding='utf-8') as file_in:
    for line in file_in:
        print(line.strip())

Reading CSV or JSON Data from .gz Files (using pandas)

If your .gz file contains structured data like CSV or JSON, the pandas library provides convenient functions to read them directly:

import gzip  # Still needed for decompression
import pandas as pd

with gzip.open('example.csv.gz', 'rt', encoding='utf-8') as file_in: # 'rt' for text mode
    df = pd.read_csv(file_in)
    print(df.head())

with gzip.open('example.json.gz', 'rt', encoding='utf-8') as file_in:
    df = pd.read_json(file_in, lines=True)  # Read line-delimited JSON
    print(df.head())

with gzip.open('example.json.gz', 'rb') as file_in:
    data = json.load(file_in)               # Read data from json file.
    df = pd.DataFrame(data)                 # Create dataframe.
    print(df.head())

• CSV: pd.read_csv() can directly read from a file-like object, so we pass the gzip.open() result to it.
• JSON (lines=True): For line-delimited JSON, use pd.read_json(..., lines=True).
• JSON (Array) If the .gz file contains a single, large JSON array, use json.load() to load the data into a Python object, and then convert it into a DataFrame.