sqlglot: Instructions file for Codex

Contributing to SQLGlot

About SQLGlot

SQLGlot is a no-dependency SQL parser, transpiler, optimizer, and engine written in pure Python. It supports 31+ SQL dialects and can transpile between them while preserving semantics. The codebase is performance-critical despite being pure Python, with an optional mypyc-compiled C extension for speed improvements (sqlglotc/).

Development Commands

Before contributing read CONTRIBUTING.md

Installation

# Basic installation
make install

# Development installation (Python only, no C extension)
make install-dev

# Development installation with mypyc C extension
make install-devc

# Install pre-commit hooks
make install-pre-commit

# With uv (faster):
UV=1 make install-dev

Testing

# Run all tests (pure Python, hides .so files during run)
make test

# Run all tests with mypyc C extension (builds extension first)
make testc

# Run only unit tests (skip integration tests, pure Python)
make unit

# Run only unit tests with C extension
make unitc

# Run specific test file
python -m unittest tests.test_expressions

# Run specific test class
python -m unittest tests.test_expressions.TestExpressions

# Run specific test method
python -m unittest tests.test_expressions.TestExpressions.test_alias

Linting & Type Checking

# Run linter and formatter only
make style

# Run full checks (style + pure Python tests + C extension tests)
make check

Benchmarks

# Run parsing benchmark
make bench

# Run optimization benchmark
make bench-optimize

Architecture Overview

SQLGlot follows a classic compiler architecture with three main phases:

1. Tokenizer (tokens.py)

• Converts SQL strings into a sequence of tokens (lexical analysis)
• Pure Python implementation in tokens.py; core logic in tokenizer_core.py (mypyc-compiled when using [c] extra)
• Maps lexemes to TokenType enum values via KEYWORDS and SINGLE_TOKENS dictionaries
• Dialects can override tokenizer behavior by customizing these mappings