AutoRAG-Research: Instructions file for Codex

AGENTS.md

Project Overview

AutoRAG-Research is a Python framework for automating RAG (Retrieval-Augmented Generation) research workflows. It provides tools for data ingestion, pipeline execution, and evaluation metrics. The user can download pre-ingested datasets and run pre-made RAG pipelines and evaluate it. The pipelines can be customized. There are two types (retrieval/generation) pipelines.

Common Commands

# Setup
make install              # Create venv, install deps, setup pre-commit hooks
uv sync --all-groups --all-extras  # Install all deps including optional (gpu, search)

# Code Quality
make check               # Run all checks (ruff, ty type checker, deptry)

# Testing (requires Docker)
make test                # Full test with Docker PostgreSQL lifecycle management
make test-only           # Run tests (assumes PostgreSQL container is running)
make test-data          # Run tests only marked as data
make test-full         # Run all tests including api/gpu/data marked tests

# Run single test
uv run pytest tests/path/to/test_file.py::test_function_name -v

# Docker
make docker-up           # Start PostgreSQL container
make docker-wait         # Wait for PostgreSQL readiness
make docker-down         # Stop container
make clean-docker        # Remove container and volumes

# Docs
make docs                # Build and serve docs locally

Architecture

The codebase follows a layered architecture with Generic Repository + Unit of Work + Service Layer patterns:

Executor/Evaluator (config.py, executor.py, evaluator.py)
    ↓
Pipeline Layer (pipelines/)
    ↓
Service Layer (orm/service/) - Business logic
    ↓
Unit of Work (orm/uow/) - Transaction management
    ↓
Repository Layer (orm/repository/) - Data access (GenericRepository[T])
    ↓
ORM Models (orm/models/) - SQLAlchemy with pgvector

Pipeline Types:

• Retrieval Pipelines (pipelines/retrieval/) - Vector search, BM25, hybrid retrieval
  • Extend BaseRetrievalPipeline
  • Use RetrievalPipelineService + RetrievalUnitOfWork
  • Methods: .retrieve(query, top_k) for single-query, .run() for batch
• Generation Pipelines (pipelines/generation/) - LLM-based answer generation
  • Extend BaseGenerationPipeline
  • Use GenerationPipelineService + GenerationUnitOfWork
  • Compose with retrieval pipelines for flexible RAG strategies
  • Example: NaiveRAGPipeline (single retrieve + generate)