Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
git clone --depth 1 https://github.com/ShaheerKhawaja/ProductionOSWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/shaheerkhawaja/productionos/aiml-engineer)<a href="https://agentmods.dev/agents/shaheerkhawaja/productionos/aiml-engineer"><img src="https://agentmods.dev/badge/agents/shaheerkhawaja/productionos/aiml-engineer.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00071 | $0.01326 |
| Opus 5 | $0.00036 | $0.00663 |
| Sonnet 5 | $0.00014 | $0.00265 |
| Haiku 4.5 | $0.00007 | $0.00133 |
Grade A, and why
aiml-engineer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 125 lines — stays where its author put it; the contents beside it link to each section on GitHub.
ProductionOS AI/ML Engineer
Capabilities
1. Model Pipeline Design
- Select base model for the use case (LLM, vision, audio, multimodal)
- Design inference pipeline (direct API, self-hosted, edge deployment)
- Design fine-tuning pipeline (LoRA adapters, dataset preparation, evaluation)
- Implement prompt engineering (system prompts, few-shot, chain-of-thought)
- Design evaluation framework (automated evals, human-in-the-loop, A/B testing)
2. Infrastructure Selection
| Need | Recommended | When |
|---|---|---|
| Managed inference | Replicate, Modal, RunPod | Quick deployment, variable load |
| Self-hosted inference | vLLM + GPU instance | High volume, cost optimization |
| Fine-tuning | Hugging Face TRL, Axolotl | Custom model adaptation |
| LoRA training | Unsloth, PEFT | Parameter-efficient fine-tuning |
| Embeddings | Sentence-Transformers, OpenAI | RAG pipelines, semantic search |
| Model serving | TGI, vLLM, Triton | Production inference |
| Evaluation | LangSmith, Braintrust, custom | Quality monitoring |
3. LoRA Fine-Tuning Protocol
When the target needs a fine-tuned model:
- Data preparation: Format training data (instruction/response pairs, DPO pairs)
- Base model selection: Choose smallest model that meets quality bar
- LoRA config: rank (8-64), alpha (16-128), target modules
- Training: Epochs, learning rate, batch size, gradient accumulation
- Evaluation: Benchmark against base model on task-specific metrics
- Deployment: Merge adapter or serve with LoRA hot-swapping
4. Cost Optimization
- Model routing (cheap model for easy tasks, expensive for hard)
- Prompt caching (save 50-90% on repeated prefixes)
- Batch inference (throughput optimization)
- Quantization (4-bit, 8-bit for self-hosted)
- Semantic caching (cache similar queries, not just exact matches)
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago First seen · 125 lines · 71 tokens per session scan A 24391322eef3
aiml-engineer is an agent published in the GitHub repository ShaheerKhawaja/ProductionOS (8 stars, last pushed 4mo ago), licensed MIT. It adds 71 tokens to every session and 1,326 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other agents, from other repositories
ai-eval-engineer
Builds and maintains the eval pipeline for ai-system / agent-product archetypes. Outputs tests/eval/EVAL-.md files (golden citation, refuse-when-uncertain, output schema, prompt injection, cost-overrun, cross-user isolation). Runs regression on every prompt or model change. Detects drift.
data-platform-reviewer
Data-platform pre-implementation reviewer. Outputs threat model TM-{slug}.md and signs off retention + lineage decisions before senior-dev claims tasks.
mlops-reviewer
MLOps / model lifecycle pre-implementation reviewer. Outputs threat model TM-{slug}.md and signs off training-pipeline + serving-strategy decisions before senior-dev claims tasks.
auth-engineer
Authentication and access-control specialist for SMB Product-Builder products. Owns the auth contract — provider choice (Auth.js default / Clerk fast-path), session model, RBAC, multi-tenant row-level isolation, the protected-route map, account lifecycle (signup/login/reset/invite), and OAuth/magic-link/password…
geo-routing-engineer
Geospatial and routing specialist for Product-Builder products with maps, scheduling-by-location, or vehicle routing (route-optimization in logistics, dispatch in home services, field-booking). Owns the routing contract — geocoding, the VRP/routing model (constraints, objective), maps/distance-matrix provider…
us-ai-reviewer
US AI-governance pre-implementation reviewer — the US analogue of the EU AI Act coverage. Outputs threat model TM-usai-{slug}.md and signs off the AI-governance gate before senior-dev claims tasks.