What is analyzing-llm-rationale AGENTS.md?

Repository instructions for a batch system that evaluates language-model reasoning on yes-or-no forecasting questions. It includes setup commands, required environment variables, tests, linting, and key commands.

Use it when configuring, testing, linting, or running the analyzing-llm-rationale project and its batch inference or FastAPI server.

It gives an agent the project-specific information needed to install dependencies, run the system, and check changes correctly. It also explains the main pipeline and serving setup.

How pareelamre/analyzing-llm-rationale configures Codex and OpenCode

Reuse

Borrowing it

Nothing to install: this file belongs to pareelamre/analyzing-llm-rationale. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file

curl -O https://raw.githubusercontent.com/pareelamre/analyzing-llm-rationale/main/AGENTS.md

Clone the repo

git clone --depth 1 https://github.com/pareelamre/analyzing-llm-rationale

Made for: Codex, OpenCode.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/instructions/pareelamre/analyzing-llm-rationale/agents-md/github.svg)](https://agentmods.dev/instructions/pareelamre/analyzing-llm-rationale/agents-md)

Your own site

<a href="https://agentmods.dev/instructions/pareelamre/analyzing-llm-rationale/agents-md"><img src="https://agentmods.dev/badge/instructions/pareelamre/analyzing-llm-rationale/agents-md/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/instructions/pareelamre/analyzing-llm-rationale/agents-md"><img src="https://agentmods.dev/badge/instructions/pareelamre/analyzing-llm-rationale/agents-md.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 1,934 This file is loaded in full into every session.

When invoked 1,934 The same file — it is already loaded in full.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.01934	$0.01934
Opus 5	$0.00967	$0.00967
Sonnet 5	$0.00387	$0.00387
Haiku 4.5	$0.00193	$0.00193

Measured 7d ago against content hash 753966596471, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-08, from the pricing page.

Security

Grade A, and why

analyzing-llm-rationale AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

AGENTS.md · 172 lines

How it starts

The opening of the file, as written. The whole thing — 172 lines — stays where its author put it; the contents beside it link to each section on GitHub.

AGENTS.md — Codex agent setup guide

Repository overview

Batch inference system for evaluating LLM reasoning on binary forecasting questions (Metaculus dataset). The pipeline runs 9 prompt variants across multiple models, stores results as JSON, and exposes a FastAPI server deployed to GCP Cloud Run and Vertex AI.

Environment setup

# Install core + serving + pipeline dependencies
pip install -e ".[dev,serve,pipeline]"

# Required environment variables
export SCADS_AI_API_KEY=<key>       # SCADS AI — used by all hosted models
export NEWSAPI_KEY=<key>            # Optional — improves news fetch quality

Use Python 3.10+ for local development. If pip install -e . fails (editable install issue with system Python), sync source files manually:

cp src/analyzing_llm_rationale/*.py \
   /home/paam844f/.local/lib/python3.10/site-packages/analyzing_llm_rationale/

Or prefix commands with PYTHONPATH=src.

Running tests and lint

python -m unittest discover -s tests   # unit tests
ruff check src tests                   # lint (E501 is ignored)

Always run both before committing.

Key CLI commands

# Batch inference
analyze-llm-rationale run-batch \
  --variant variant0_neutral_baseline \
  --model gpt-oss-120b \
  --temperature 0.0 --temperature-tag temperature_00

# Start API server locally (Note: Port 8000 is reserved, run on 8080 instead)
PYTHONPATH=src python -m uvicorn analyzing_llm_rationale.server:app --port 8080

# Fetch + rank news for a question (LangChain pipeline)
PYTHONPATH=src analyze-llm-rationale fetch-and-rank \
  --question "Will X happen by date Y?"

# DuckDB analytics — ingest all results and run 10 SQL queries
python scripts/sql_analytics.py --ingest

# Prefect pipeline — fetch news, run inference, store in DuckDB
python flows/forecasting_flow.py --question-id 124

SLURM submission rule

Submit SLURM jobs only from the /data/horse/ws/... workspace, not from /home. Use --chdir or run sbatch with the working directory set to the project path under /data/horse/ws so logs, outputs, and temporary files stay off the home quota.

Read the full file on GitHub · 172 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

7d ago First seen · 172 lines · 1,934 tokens per session scan A 753966596471

Subscribe to this mod's changes

analyzing-llm-rationale AGENTS.md is an instructions file published in the GitHub repository pareelamre/analyzing-llm-rationale (0 stars, last pushed today), licensed MIT. It adds 1,934 tokens to every session, about $0.0097 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

next.js AGENTS.md

AGENTS.md instructions for vercel/next.js, covering next.js development guide, codebase structure, monorepo overview, core package: packages/next and other important packages.

vercel/next.js · 7,296 tokens

codex AGENTS.md

AGENTS.md instructions for openai/codex, covering rust/codex-rs, the codex-core crate, code review rules, crate api surface and model visible context.

openai/codex · 5,153 tokens

vscode buildNext.instructions.md

Working notes and architecture documentation for the new esbuild-based build system in build/next. Use when making changes to the new build pipeline (transpile/bundle commands, NLS plugin, source-map handling, resource copying, or self-hosting watch tasks).

microsoft/vscode · 6,785 tokens

analyzing-llm-rationale: Instructions file for Codex