LLM Evaluation System is an agent-guided platform for evaluating language models and agents, generating datasets and configuring multiple judges from natural-language requests before producing an analysis report. It is for comparing model responses, testing agents, and creating document-grounded evaluation data.
Borrowing it
Nothing to install: this file belongs to awslabs/llm-evaluation-system. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/awslabs/llm-evaluation-system/main/.claude/skills/aws-architecture/SKILL.mdgit clone --depth 1 https://github.com/awslabs/llm-evaluation-systemWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/awslabs/llm-evaluation-system/aws-architecture)<a href="https://agentmods.dev/skills/awslabs/llm-evaluation-system/aws-architecture"><img src="https://agentmods.dev/badge/skills/awslabs/llm-evaluation-system/aws-architecture/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/awslabs/llm-evaluation-system/aws-architecture"><img src="https://agentmods.dev/badge/skills/awslabs/llm-evaluation-system/aws-architecture.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00109 | $0.01885 |
| Opus 5 | $0.00055 | $0.00942 |
| Sonnet 5 | $0.00022 | $0.00377 |
| Haiku 4.5 | $0.00011 | $0.00188 |
Grade A, and why
aws-architecture scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
- `scripts/fetch_icons.sh` needs `curl` + `unzip`. How it starts
The opening of the file, as written. The whole thing — 127 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AWS Architecture Diagram Skill
Build clean, AWS-poster-style architecture diagrams using the official AWS Architecture Icons, authored as a hand-laid-out SVG (inside an HTML wrapper) and rendered to PNG via headless Chromium. This is the approach that reliably avoids the two things that wreck these diagrams: crossing arrows and lines cutting through text.
Why SVG (not draw.io, not HTML/CSS divs)
- The hard part of an architecture diagram is the connectors (routed arrows
between nodes). SVG
<path>with explicit waypoints + a<marker>arrowhead is the only medium where you control every bend, so lines route through gaps and never cross labels. - draw.io's auto-router makes uncontrollable diagonal crossings; CSS has no line primitive (you'd fake arrows with rotated divs — looks bad).
- One SVG coordinate system means icons and line endpoints align exactly, and paint order (lines first, icons second) tucks arrows neatly behind icons.
Workflow
- Fetch icons (once):
bash scripts/fetch_icons.sh— downloads + caches the official AWS icon package undercache/aws-icons/. Idempotent. - Resolve each service icon:
python3 scripts/find_icon.py cache <name>prints the best-match SVG path (e.g.cloudfront,cognito,elastic kubernetes,rds,bedrock,simple storage,elastic load balancing). Use--list <name>to see candidates. Gotcha: general/resource icons ship in_Light(dark glyph) and_Dark(white glyph) variants — on a white canvas always use_Light(the resolver already prefers it). The "User" actor icon isRes_User_48_Light.svg. - Author the SVG (see Template + Layout rules below). Reference icons with
<image href="file:///abs/path/icon.svg" .../>while iterating. - Render:
python3 scripts/render.py diagram.html out.png(screenshots the<svg id="d">). Add--scale 2for crisp output (default 2). - CRITICAL — self-critique loop: actually Read the rendered PNG, list the worst flaw (a line crossing text, a label collision, clipping, an invisible/white icon, an off-center trunk), fix that ONE thing, re-render. Repeat until clean. This loop is the skill — do not stop after one render. Typically 4-6 passes.
- Finalize self-contained: once good, run
python3 scripts/render.py diagram.html out.png --embedto inline the icon SVGs as base64 data URIs (so the source has nofile://deps), then render the final PNG. Commit the.htmlsource + PNG together.
What ships with it
4 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 9d ago First seen · 127 lines · 109 tokens per session scan A 9e264de4253e
aws-architecture is a skill published in the GitHub repository awslabs/llm-evaluation-system (23 stars, last pushed today), licensed Apache-2.0. It adds 109 tokens to every session and 1,885 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
gke-compute-classes
Configures, optimizes, and troubleshoots GKE ComputeClasses. Use when configuring Spot VMs with on-demand fallback, targeting specific accelerators (GPUs/TPUs) or machine families, restricting ComputeClass access, or debugging pending pods related to node pool auto-creation. Do not use for cluster-level Node Auto…
gke-workload-security
Audits, configures, and hardens workload-level security controls for Google Kubernetes Engine (GKE) applications and namespaces. Covers running cluster security audits (auditcluster.sh), configuring Workload Identity Federation (impersonation, KSA/GSA binding, and pod setup), enforcing Network Policies (default-deny…
gke-reliability
Improves GKE workload reliability, using PDBs, health probes, and topology spread constraints. Use when configuring GKE workload reliability, setting up PDBs, or configuring GKE health probes (liveness, readiness, startup). Don't use for disaster recovery setup or full cluster backups (use gke-backup-dr instead).
nemo-automodel-launcher-config
Configure NeMo AutoModel job launches for interactive runs, Slurm clusters, and SkyPilot cloud execution.
azure-mgmt-botservice-dotnet
Azure Resource Manager SDK for Bot Service in .NET. Management plane operations for creating and managing Azure Bot resources, channels (Teams, DirectLine, Slack), and connection settings. Triggers: "Bot Service", "BotResource", "Azure Bot", "DirectLine channel", "Teams channel", "bot management .NET", "create bot".
cloud-architect
Designs cloud architectures, creates migration plans, generates cost optimization recommendations, and produces disaster recovery strategies across AWS, Azure, and GCP. Use when designing cloud architectures, planning migrations, or optimizing multi-cloud deployments. Invoke for Well-Architected Framework, cost…