Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add decodingai-magazine/building-a-coding-agent-from-scratch-course --skill manual-e2e-qagit clone --depth 1 https://github.com/decodingai-magazine/building-a-coding-agent-from-scratch-courseWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/decodingai-magazine/building-a-coding-agent-from-scratch-course/manual-e2e-qa)<a href="https://agentmods.dev/skills/decodingai-magazine/building-a-coding-agent-from-scratch-course/manual-e2e-qa"><img src="https://agentmods.dev/badge/skills/decodingai-magazine/building-a-coding-agent-from-scratch-course/manual-e2e-qa.svg" alt="Measured on agentmods" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 3 findings, up to high
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- high Privilege Escalation · line 15 Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.Fix: Remove references to credential paths. Use environment variables or secrets managers. For docs, use placeholder paths (e.g., /path/to/config). Never load .env or token files in production code paths.
- high Privilege Escalation · line 19 Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.Fix: Remove references to credential paths. Use environment variables or secrets managers. For docs, use placeholder paths (e.g., /path/to/config). Never load .env or token files in production code paths.
- high Tool Misuse · line 49 Tool parameters are crafted to achieve unintended or unsafe behavior. Parameter abuse can bypass intended safety checks (e.g. shell=True, --force, dangerous glob patterns).Fix: Validate all tool parameters against an allowlist. Reject dangerous parameter values (shell=True, --force, -rf /) and use safe defaults.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00108 | $0.05016 |
| Opus 5 | $0.00054 | $0.02508 |
| Sonnet 5 | $0.00022 | $0.01003 |
| Haiku 4.5 | $0.00011 | $0.00502 |
Grade C, and why
manual-e2e-qa scanned grade C with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Recursive force deletehighDestructive command
rm -rf with a variable or a broad path is one typo away from removing the wrong tree.
| **sandbox git token (`SANDBOX_GIT_TOKEN`, both backends)** | `rm -rf .decode/sandbox` (**required** — see below), then `SANDBOX_GIT_TOKEN=<fine-grained-PAT> SANDBOX_MODE=docker decode run --repo <url> "create NOTES.md How it starts
The opening of the file, as written. The whole thing — 61 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Testing E2E
Automated M1 proof: capstone tests/integration/test_milestone1_capstone.py — six-step conversation (read → gated write approve → gated write deny → todo_write → ask_user → web_fetch) through real build_agent() + Runner + render_event + session log + memory write-back; only network boundary swapped (FunctionModel for model, httpx.MockTransport for web tool). Run: make integration-tests or make ci. No API key, no network.
Below: manual e2e vs real Gemini — exercise each surface, then try to break it (adversarial half = Tester's job).
Launch (one env var, no service):
export GEMINI_API_KEY=… # the only required secret (see .env.example); or put it in .env
uv run decode # the REPL: a "> " prompt + a footer hint render
No GEMINI_API_KEY → one friendly stderr line — decode: set GEMINI_API_KEY in your environment or .env to start (see .env.example). — exit non-zero, no traceback (task-004 guard in cli.py).
Per surface: what to type + what "working" looks like.
| Surface | Type this | Working looks like |
|---|---|---|
| Plain chat | what can you do? |
answer streams token-by-token above prompt; prompt pinned at bottom. |
| Read (gated) | read pyproject.toml |
permission? read … prompt; y → panel with numbered file contents; n → model told denied, adapts. |
| Write (gated, approve) | create a file hello.txt that says hi |
permission? write … → y → file appears on disk (cat hello.txt), model confirms. |
| Write (gated, deny) | repeat write, answer n |
file not created (ls hello.txt → absent); model told denied (doesn't pretend it wrote). |
| Bash | run the tests with make unit-tests |
permission? bash … → y → panel with stdout/stderr (truncated past cap); runaway command bounded by bash_timeout_s. |
| Todo checklist | make a 3-step plan to add a CLI flag and track it |
blue tasks panel renders checklist ([ ] / [~] / [x]), re-renders as statuses update. |
| web_fetch | fetch https://example.com and summarize it |
permission? web_fetch … → y → page as Markdown (HTML stripped), model summarizes. |
| ask_user | deploy my app (underspecified) |
model calls ask_user; ask: … question + type your answer: cue; next typed line is the answer, turn resumes. |
| lsp (code intelligence) | where is build_agent defined? |
model calls lsp (definition); auto-allows (read-only, no prompt); answer cites src/decode/agent/factory.py:68:5. Then write a broken bad.py with a syntax error → approved write's result carries appended LSP diagnostics (ty) — fix these: block; model corrects. |
| agent (Explore subagents) | explore how permissions and sandboxing each work across the repo, in parallel |
ONE agent(prompts=[…]) call — one prompt per angle — auto-allows (READ_ONLY — "can only cause reads"); result panel = ONE labelled aggregate, a ## Subagent i — "<prompt>" section per child, in prompt order (no synthesis LLM call — the parent model synthesizes). Child = read-only Explore subagent (read/glob/grep/lsp only — no write/edit/bash/web_fetch). The harness guarantees the parallelism (asyncio.gather inside the tool, not the model volunteering N calls): width cap 6 prompts per call (ModelRetry → "consolidate" above it; empty list → ModelRetry), concurrency capped subagent_max_parallel (default 4 — a 6-wide fan-out runs 4, then 2). Child bounded subagent_max_requests (default 25); reports share the fold budget — each truncated to subagent_result_max_bytes // len(prompts) (default 16000 total, so the fold costs the same at any width). Input guarded pre-spawn: a prompt under 8 words (explore the repo) nags the model back (ModelRetry naming Prompt i ("…") as too terse) and spawns NOTHING — rewrite and it fans out (a nag costs one retry leg; agent registers retries=3, so a stubborn model is coached, not aborted). Output validated: a report that is empty or backed by zero tool calls (child answered from memory) buys that child exactly one re-spawn with a nudge — two attempts, never three — then its section carries The subagent returned no usable report.; a child that raises gets This subagent failed before producing a report. Either way the siblings still fold. Every result ends with the Synthesis Footer — the harness telling the parent to compile the sections into ONE answer: prose + a text diagram (ASCII/box-drawing; Mermaid only for a genuine graph), so a good fan-out answer shows you a diagram in the terminal. Children silent-until-done (no-op event sink), transcripts ephemeral — --resume carries only the one spawn call + the folded aggregate (ADR-0013, ADR-0017). |
| Opik tracing (observability) | export OPIK_API_KEY=<comet-key> (free at comet.com), relaunch uv run decode, run any turn; open it in Opik UI |
on launch Decode - Opik tracing on (project 'decode-local'). prints once, before banner — the project is DERIVED decode-<DECODE_ENV> unless OPIK_PROJECT_NAME is set (ADR-0015 §8), so DECODE_ENV=prod files traces under decode-prod; unset key → byte-identical (zero spans, no network, no line). One REPL turn = one Trace (root Span chat_turn); session's traces = one Thread keyed on session id — gated tool's approve/resume leg + follow-ups ride the same trace (latency honestly includes gate wait). Every LLM + tool call = Span: inputs/outputs, latency, tokens (gen_ai.usage.*), cost for models Opik prices (Gemini yes; OpenRouter/Modal may be tokens-only). Memory write-back + compaction ride the one global instrument_pydantic_ai(); subagent agent(...) child nests inside parent turn's trace, child tokens visible (closes ADR-0013 §9). --resume mints fresh session id → new Thread. Headless: a decode run = one Trace (decode_run / decode_run_hitl), Thread = Kitaru exec_id; activation surfaces only in LOG — stdout stays exactly the answer (pipe-clean), stderr untouched. --hitl pause closes run's span, resume opens fresh one under same Thread; real provider on checkpoint_strategy="calls" may export some model spans as siblings of run root (documented ceiling — tokens ride every span). Evals/experiments = M13 (ADR-0014). |
| decode run (headless) | decode run "list the python files" (separate command, not in REPL) |
agent tool-loops headlessly through Kitaru durable flow — every tool inline under bypass, no prompt — prints result, exit 0. Recorded as inspectable checkpointed execution; fresh re-run = new execution (crash-resume replay exercised in 059 / capstone). RUNTIME_ENABLED=false → one friendly stderr line, non-zero exit, no flow built (ADR-0008). |
| decode run --hitl (durable HITL) | decode run --hitl "create config.toml, then deploy"; resolve from a second terminal |
read-only tools run inline; write/edit/bash (or ask_user/exit_plan_mode) pauses whole execution on durable Kitaru wait. While it polls: kitaru executions list → waiting <exec_id> + wait <name>; then kitaru executions input <exec_id> --wait <name> --value 'true' (approve), 'false' (deny → run stops, tool never ran), or '"staging"' (ask_user answer). Run resumes, prints result. Unanswered wait times out → run pauses, prints <exec_id> + kitaru executions input hint, exit 0. Timeout differs by wait kind (known limitation — decode doesn't fork adapter): ask_user/exit_plan_mode answer waits honor runtime_wait_timeout_s; native write/edit/bash approval waits use adapter's fixed 600s, ignore the setting (ADR-0008 §3). |
| decode run --model (model override) | decode run --model gemini-2.5-pro "list the python files" |
same bypass run; Model Override swaps only active provider's model id for this run (provider stays LLM_PROVIDER-selected — no cross-provider swap; ADR-0010 §2). Answer → stdout (pipe-clean); stderr → durable exec_id: <id> + paste-ready replay it with a change: decode replay <id> --model gemini-2.5-pro hint. Presence, not correctness — wrong model id for provider fails at first model request. Model rides as durable flow input → later decode replay can swap it (ADR-0010 §4). |
| decode replay --model (what-if replay) | keep an exec_id from a decode run, then decode replay <exec_id> --from <checkpoint> --model gemini-2.5-pro |
re-executes that recorded bypass run from --from with model swapped: turns before --from serve from original run's cache, anchor + downstream re-execute — swap bites only downstream (ADR-0010 §5). Answer → stdout; stderr → new Fork exec_id:, original: id, compare them: kitaru executions get <new> vs kitaru executions get <original> diff hint. --from required (no default anchor); omitted → one friendly line (checkpoints via kitaru executions get <exec_id>; --model omitted = replay as-is). Bypass-only: HITL exec_id refused with friendly line pointing at kitaru executions replay <id> (HITL replay re-asks every wait — ADR-0010 §5,7; answer-reuse deferred, tasks/future/hitl-replay-answer-reuse.md). Ambiguous/invalid --from or diverged swap → one friendly line, non-zero, never traceback. Offline-provable: tests/integration/test_runtime_capstone.py::test_model_swap_replay_re_executes_downstream_turns; deferred HITL answer-reuse needs deployed stack. |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 8d ago First seen · 61 lines · 108 tokens per session scan C ef49f4c0a433
manual-e2e-qa is a skill published in the GitHub repository decodingai-magazine/building-a-coding-agent-from-scratch-course (368 stars, last pushed 4d ago), licensed Apache-2.0. It adds 108 tokens to every session and 5,016 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it C with 1 finding (recursive force delete). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
write-e2e-test
Write an end-to-end (e2e) test for vix. Use when the user asks to add, create, or modify an e2e test that drives the real vix TUI and vixd daemon against the mock LLM server.
playwright-automation
Browser automation and testing with Playwright. Use when testing web applications, automating browser tasks, taking screenshots, or validating UI behavior.
e2e-testing
ORGII keeps two separate E2E surfaces. Do not use one as proof for the other.
dual-instance-verification
Dual-instance (双机) real-machine verification protocol for ORG2 cloud sync and session sharing. Use before declaring any sharing/sync/collab feature or fix "verified": share/unshare, push/retract, fork/import, comments, member-floor, replay, continuation, or anything touching Org2CloudSyncEngine, collab engines, or the…
drive-automation-session
Drive an already-reserved Kobiton device from a natural-language intent. Opens an automation Appium session directly against the Kobiton WebDriver hub, runs an observe-decide-act loop with one action per iteration, pauses to ask the user when stuck (same-action repetition, screen unchanged, or model self-declared…
monitor-test-run
Watch a running Kobiton test run and narrate it to the user: read the org's live-remediation flag up front, poll the run until every execution is terminal, surface the live-remediation URL the moment an execution is blocked, and give a correct post-mortem so a COMPLETED-with-BLOCKERENCOUNTERED execution is never…