How to Convert Escaped String to JSON in Python

Learn how to parse and convert double-encoded, HTML-escaped, URL-encoded, or Unicode-escaped JSON strings in Python using json.loads(), html.unescape(), and other techniques.

APIs and web services often return JSON that has been "double-encoded" or escaped within another string. This commonly occurs in nested API responses, AWS Lambda logs, HTML attributes, or when JSON is embedded in other formats.

This guide covers techniques for unescaping and parsing these problematic JSON strings.

Standard JSON String Parsing

For regular JSON strings, use json.loads() directly:

import json

raw_string = '{"user": "Alice", "count": 5, "active": true}'

data = json.loads(raw_string)

print(data['user'])    # Alice
print(data['count'])   # 5
print(type(data))      # <class 'dict'>

Double-Escaped JSON

When JSON is stringified inside another JSON string, you'll see double backslashes. This requires parsing twice:

import json

payload = '"{\\"id\\": 123, \\"name\\": \\"test\\", \\"active\\": true}"'

intermediate = json.loads(payload)
print(intermediate)        # {"id": 123, "name": "test", "active": true}
print(type(intermediate))  # <class 'str'> - Still a string!

final_data = json.loads(intermediate)
print(final_data['id'])    # 123
print(type(final_data))    # <class 'dict'>

Detecting Double-Encoded JSON

import json

def parse_json_flexible(data):
    """Parse JSON, handling double-encoding automatically."""
    if isinstance(data, str):
        parsed = json.loads(data)
        
        if isinstance(parsed, str):
            return json.loads(parsed)
        return parsed
    
    return data

normal = '{"key": "value"}'
double = '"{\\"key\\": \\"value\\"}"'

print(parse_json_flexible(normal))   # {'key': 'value'}
print(parse_json_flexible(double))   # {'key': 'value'}

HTML Entity Escaped JSON

JSON embedded in HTML attributes uses HTML entities like " for quotes:

import html
import json

meta_content = '{&quot;lat&quot;: 40.7128, &quot;lon&quot;: -74.0060, &quot;city&quot;: &quot;New York&quot;}'

clean_json = html.unescape(meta_content)
print(clean_json)

location = json.loads(clean_json)
print(f"City: {location['city']}")

Common HTML Entities in JSON

Entity	Character	Description
`"`	`"`	Double quote
`'`	`'`	Single quote
`&`	`&`	Ampersand
`<`	`<`	Less than
`>`	`>`	Greater than

Unicode Escape Sequences

JSON may contain Unicode escape sequences (\uXXXX):

import json

escaped = '{"message": "Hello \\u0048\\u0065\\u006c\\u006c\\u006f", "emoji": "\\ud83d\\ude00"}'

data = json.loads(escaped)
print(data['message'])  # Hello Hello
print(data['emoji'])    # 😀

URL-Encoded JSON

JSON passed in URLs or query strings may be URL-encoded:

from urllib.parse import unquote
import json

url_encoded = '%7B%22user%22%3A%20%22Alice%22%2C%20%22id%22%3A%20123%7D'

decoded = unquote(url_encoded)
print(decoded)

data = json.loads(decoded)
print(data['user'])

Python String Literals (ast.literal_eval)

Sometimes you receive Python dictionary syntax instead of JSON:

import ast
import json

python_literal = "{'name': 'Alice', 'active': True, 'count': None}"

data = ast.literal_eval(python_literal)
print(data['name'])    # Output: Alice
print(data['active'])  # Output: True

Comprehensive Parsing Function

import json
import html
from urllib.parse import unquote
import ast

def parse_escaped_json(data, max_depth=3):
    """
    Parse JSON from various escaped formats.
    
    Handles:
    - Standard JSON strings
    - Double/triple encoded JSON
    - HTML entity escaped JSON
    - URL encoded JSON
    - Python literal syntax
    """
    if not isinstance(data, str):
        return data
    
    original = data
    
    if '%' in data:
        data = unquote(data)
    
    if '&' in data and ';' in data:
        data = html.unescape(data)
    
    for _ in range(max_depth):
        try:
            parsed = json.loads(data)
            
            if isinstance(parsed, str):
                data = parsed
                continue
            
            return parsed
            
        except json.JSONDecodeError:
            break
    
    try:
        return ast.literal_eval(original)
    except (ValueError, SyntaxError):
        pass
    
    raise ValueError(f"Could not parse: {original[:100]}...")

test_cases = [
    '{"key": "value"}',                              # Standard
    '"{\\"key\\": \\"value\\"}"',                    # Double-encoded
    '{&quot;key&quot;: &quot;value&quot;}',          # HTML entities
    '%7B%22key%22%3A%20%22value%22%7D',              # URL-encoded
    "{'key': 'value'}",                              # Python literal
]

for test in test_cases:
    result = parse_escaped_json(test)
    print(f"Input type -> {type(result).__name__}: {result}")

Output:

Input type -> dict: {'key': 'value'}
Input type -> dict: {'key': 'value'}
Input type -> dict: {'key': 'value'}
Input type -> dict: {'key': 'value'}
Input type -> dict: {'key': 'value'}

Debugging Escaped Strings

When you're unsure why parsing fails, inspect the raw string:

import json

problematic = '{"message": "Hello\\nWorld"}'

print(repr(problematic))

print([char for char in problematic[:20]])

try:
    data = json.loads(problematic)
except json.JSONDecodeError as e:
    print(f"Error at position {e.pos}: {e.msg}")
    print(f"Context: ...{problematic[max(0,e.pos-10):e.pos+10]}...")

Output:

'{"message": "Hello\\nWorld"}'
['{', '"', 'm', 'e', 's', 's', 'a', 'g', 'e', '"', ':', ' ', '"', 'H', 'e', 'l', 'l', 'o', '\\', 'n']

Summary

Input Format	Solution	Example
`{"key": "val"}`	`json.loads(s)`	Standard JSON
`"{\"key\": ...}"`	`json.loads()` twice	Double-encoded
`{"key"}`	`html.unescape()` + `json.loads()`	HTML entities
`%7B%22key%22...`	`unquote()` + `json.loads()`	URL-encoded
`{'key': 'val'}`	`ast.literal_eval(s)`	Python literal