sqrl: Instructions file for Codex

DataSQRL

This file provides guidance to coding agents (e.g., Claude Code, Codex) when working with code in this repository.

Project Overview

DataSQRL is an open-source data engineering harness for building data engineering agents designed around human control, correctness, and safety. It extends coding agents with a SQL compiler, a validator, an event-time simulator, and skills. The compiler turns SQL-like scripts (SQRL) into complete data pipelines that integrate Kafka, Flink, PostgreSQL, Iceberg, GraphQL APIs, and LLM tooling. Built with Java 17 and Maven.

External Dependencies

The following repositories contain additional runtime components:

• Flink SQL Runner: Runs the Flink compiled plan and provides additional utilities for Flink

Essential Commands

Build Commands

# Full build and test
mvn clean install

# Build with snapshot updates
mvn clean install -P update-snapshots

# Quick build (skip tests and checks)
mvn clean install -P quickbuild

# Format code automatically
mvn -P dev initialize

# Server-specific builds
mvn clean package                    # Build fat JAR (vertx-server.jar)
mvn clean package -Pskip-shade-plugin  # Build without fat JAR

Testing Commands

# Unit tests only
mvn test

# Integration tests
mvn verify

# Test specific module
mvn test -pl sqrl-planner

# Test specific test method in module
mvn test -pl sqrl-tools/sqrl-config -Dtest=TestClassName#testMethodName

# Container tests (requires Docker images to be built)
mvn -B install -DonlyContainerE2E -pl :sqrl-testing-container -Dit.test=TestClassName

# Container tests with dev profile (recommended)
mvn -B install -Pdev -Dit.test=TestClassName

# Run all container tests (omit -pl when testing container code changes)
mvn -B install -DonlyContainerE2E -Dit.test=*ContainerIT

Code Quality

# Check code formatting
mvn validate-code-format

# Format code (Google Java Format)
mvn -P dev initialize