python-json-parsing

Python JSON Parsing Best Practices

Comprehensive guide to JSON parsing in Python with focus on performance, security, and scalability.

Quick Start

Basic JSON Parsing

import json

# Parse JSON string
data = json.loads('{"name": "Alice", "age": 30}')

# Parse JSON file
with open("data.json", "r", encoding="utf-8") as f:
    data = json.load(f)

# Write JSON file
with open("output.json", "w", encoding="utf-8") as f:
    json.dump(data, f, indent=2)

Key Rule: Always specify encoding="utf-8" when reading/writing files.

When to Use This Skill

Use this skill when:

• Working with JSON APIs or data interchange
• Optimizing JSON performance in high-throughput applications
• Handling large JSON files (> 100MB)
• Securing applications against JSON injection
• Extracting data from complex nested JSON structures

Performance: Choose the Right Library

Library Comparison (10,000 records benchmark)

Library	Serialize (s)	Deserialize (s)	Best For
orjson	0.42	1.27	FastAPI, web APIs (3.9x faster)
msgspec	0.49	0.93	Maximum performance (1.7x faster deserialization)
json (stdlib)	1.62	1.62	Universal compatibility
ujson	1.41	1.85	Drop-in replacement (2x faster)

Recommendation:

• Use orjson for FastAPI/web APIs (native support, fastest serialization)
• Use msgspec for data pipelines (fastest overall, typed validation)
• Use json when compatibility is critical

Installation

# High-performance libraries
pip install orjson msgspec ujson

# Advanced querying
pip install jsonpath-ng jmespath

# Streaming large files
pip install ijson

# Schema validation
pip install jsonschema

Large Files: Streaming Strategies

For files > 100MB, avoid loading into memory.

Strategy 1: JSONL (JSON Lines)

Convert large JSON arrays to line-delimited format:

# Stream process JSONL
with open("large.jsonl", "r") as infile, open("output.jsonl", "w") as outfile:
    for line in infile:
        obj = json.loads(line)
        obj["processed"] = True
        outfile.write(json.dumps(obj) + "\n")