testing

Test-writing guidelines for a command-line tool, with a focus on integration tests that run against realistic project fixtures. Integration tests check that several parts work together from start to finish.

In plain words
What is it for?
Writing tests in tests/test_integration.py for linting and fixing fixture projects through the CLI. They also cover cases involving custom-rule imports and Unicode output.
Why use it?
They provide a consistent way to test the command line without the delay of starting a new program for every test. They also identify when a real separate process is needed for import isolation or output-encoding checks.

Cursor rule for Cursor

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add rules/stbenjam/skillsaw/testing
Clone the repo
git clone --depth 1 https://github.com/stbenjam/skillsaw

Made for: Cursor.

Per session 11 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 661 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 1 finding. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00011 $0.00661
Opus 5 $0.00005 $0.00331
Sonnet 5 $0.00002 $0.00132
Haiku 4.5 $0.00001 $0.00066

Measured 2d ago against content hash 2b2164b556f5, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

testing scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Runs shell commandslowCapability

Expected in a hook, worth knowing in a rule or an instructions file.

`subprocess.CompletedProcess` stand-in. Spawning an interpreter per call cost
.cursor/rules/testing.mdc · 65 lines

How it starts

The opening of the file, as written. The whole thing — 65 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Testing

Integration Tests

Integration tests live in tests/test_integration.py and verify end-to-end behavior by driving the CLI (skillsaw lint / fix) against realistic repo fixtures.

Drive the CLI through run_cli

run_lint and the _run_fix helpers call run_cli from tests/cli_runner.py, which invokes skillsaw.cli.main() in-process with argv, stdout, stderr, env and cwd swapped, and returns a subprocess.CompletedProcess stand-in. Spawning an interpreter per call cost ~160ms — more than half of it importing skillsaw — and dominated the suite. Prefer run_cli for a new integration test.

Reach for a real subprocess.run only when the test depends on something a shared interpreter cannot give it:

  • Import isolation — the --no-custom-rules tests prove a rule file on disk did or did not get imported. In-process they would execute an arbitrary rule inside the test worker, and sys.modules could mask a repeat import and make the negative case pass vacuously.
  • A real encoding on stdout — the lone-surrogate regressions assert that rendering a report raises no UnicodeEncodeError. A StringIO never encodes, so in-process they would pass vacuously.
  • TTY and colour-cascade behavior, which depends on the real stream.
  • Entry-point discoverytests/test_integration_plugins.py installs plugins and needs a fresh interpreter to pick them up, so it stays on subprocess throughout.

Anything run_cli runs shares one interpreter, so a helper that leaves process-global state behind breaks later tests. run_cli already resets the two that outlive a call — root logging handlers and the skillsaw.utils file cache; add to _reset_process_globals if you introduce another.

Use test fixtures

Prefer static fixtures in tests/fixtures/ over building repos programmatically in test code. Fixtures should contain realistic file content that mirrors what real users would write.

Read the full file on GitHub · 65 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 65 lines · 11 tokens per session scan A 2b2164b556f5

Subscribe to this mod's changes

testing is a cursor rule published in the GitHub repository stbenjam/skillsaw (59 stars, last pushed 2d ago), licensed Apache-2.0. It adds 11 tokens to every session and 661 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 1 finding (runs shell commands). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.