Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/microsoft/vscode/smoke-testsnpx skills add microsoft/vscode --skill smoke-testsgit clone --depth 1 https://github.com/microsoft/vscodeWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00050 | $0.02322 |
| Opus 5 | $0.00025 | $0.01161 |
| Sonnet 5 | $0.00010 | $0.00464 |
| Haiku 4.5 | $0.00005 | $0.00232 |
Grade A, and why
smoke-tests scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 151 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Running Smoke Tests
Smoke tests live in test/smoke/ and drive a full VS Code instance (Electron, web, or remote) through end-to-end user flows.
Scripts
npm run smoketest— compiles the smoke tests first (test/smoke), then runs them.npm run smoketest-no-compile— runs the already-compiled smoke tests. CI uses this after an explicit compile step.
Both forward extra arguments after -- to the runner (test/smoke/test/index.js).
Common options
| Option | Description |
|---|---|
-g <pattern> (alias -f) |
Grep filter on test/suite titles (mocha grep). |
--build <path> |
Run against a packaged build instead of the compiled-from-source dev build. |
--tracing |
Capture Playwright traces (and screenshots on failure). |
--web |
Run the browser smoke tests instead of Electron. |
--headless |
Headless browser (used with --web). |
--remote |
Run the remote smoke tests. |
# Run everything (Electron, from source)
npm run smoketest
# Run only a subset of suites by name, with tracing (replace <suite name> with your suite, e.g. "Agents Window")
npm run smoketest -- -g "<suite name>" --tracing
# Run against a packaged build (CI style)
npm run smoketest-no-compile -- --tracing --build "/path/to/VSCode-darwin-arm64/Code - OSS.app"
The -g pattern matches against test/suite titles. For example, -g "Agents Window" matches all three Agents Window suites (Agents Window, Agents Window (local AgentHost), and Agents Window (local AgentHost, SDK sandbox)); use whatever substring identifies the suite(s) you care about.
The runner exits non-zero if any test fails, so a 0 exit code means every selected test passed.
Temporarily looping a suite to hunt flaky CI tests
When a smoke test fails intermittently only in CI, a useful technique is to temporarily run the suspect suite many times in a row and fail on the first failure. This reproduces the flake under the real CI environment and captures its traces/screenshots, instead of waiting for it to recur naturally across unrelated PRs.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 151 lines · 50 tokens per session scan A ed290f30a6dd
smoke-tests is a skill published in the GitHub repository microsoft/vscode (190,384 stars, last pushed today), licensed MIT. It adds 50 tokens to every session and 2,322 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
write-ui-tests
Creates UI tests for a GitHub issue and verifies they reproduce the bug. Iterates until tests actually fail (proving they catch the issue). Use when PR lacks tests or tests need to be created for an issue.
run-device-tests
Build and run .NET MAUI device tests locally with category filtering. Supports iOS, MacCatalyst, Android on macOS; Android, Windows on Windows. Use TestFilter to run specific test categories.
browser-qa
Use when validating or debugging a workflow in the embedded browser and you need a reproducible, evidence-first loop.
verifier-gui
Launch (or attach to) the Silo dev app and drive it for runtime verification through the dev automation RPC bridge — exec commands, eval DOM, capture screenshots, and create/activate/delete workspaces, terminals, editors. This is the repo's GUI evidence-capture handle for the verify skill. Use when verifying a change…
verify
How to build, launch and drive the DeskMail AI Electron app for end-to-end verification of a change.
playwright-cli
Automate browser interactions, test web pages and work with Playwright tests.