redamon-testing

When to Use

• Writing or fixing a test anywhere in the repo, or deciding where a new test goes.
• A test is red/skipped/xfailed and you must decide whether it is real.
• You were asked to run the suite or verify a change "works".

The repo-wide rule "never validate with host pytest, use the Docker gate" lives in the root AGENTS.md CRITICAL RULES; this skill is everything after that: isolation, tiers, and how to write a test that asserts something.

Critical Rules

• NEVER run pytest across a whole tree in one process. Many tests stub langchain/langgraph into sys.modules and bake tool objects against a fake @tool at import time, so whichever file collects first decides for all of them. You get phantom failures in files you never touched (classically a coroutine was expected, got <MagicMock>). Run ./redamon.sh test, or one file / node id. The gate exists for this: tooling/scripts/pytest_isolated.py runs each FILE in its own subprocess.
• NEVER "fix" source because a test went red in a multi-file run. Re-run that one file in isolation first; if it passes alone the failure was pollution, not a bug.
• NEVER print("SKIP..."); return to skip a test. pytest records that as PASSED while asserting nothing. Use self.skipTest(...) inside a TestCase or pytest.skip(...) in a bare function.
• NEVER read a green gate as "the live checks passed". A self-skipping test that needs a service prints SKIP and exits 0, and the gate containers have no Neo4j, so the live-graph schema checks in recon/tests/test_schema_catalog.py SKIP in every CI run. Their hermetic counterparts in recon/tests/test_graph_writes_documented.py do run. Neither sees everything: the code scan cannot see the ~22 labels written with SET n += $props (the names are built in Python and appear in no file), and the live graph cannot see a feature this deployment never ran. After a schema change, run the live one against a stack before believing it.
• NEVER rewrite an assertion so it passes. If a test reveals a real bug, mark it @pytest.mark.xfail(strict=True, reason=...) and say so. Tests must not enshrine bugs.
• NEVER put a recon test in the root tests/ folder. Root tests/ runs in the agent image; recon files there must be listed in _ROOT_RECON_TESTS at redamon.sh:4898 or they run against the wrong image and fail on imports. New recon tests go in recon/tests/.
• NEVER add a third-party import to a test without checking it is in the section image. Only pytest, pytest-cov, pytest-xdist, pytest-asyncio (requirements-test.txt) are guaranteed; anything else errors the whole file at collection. Prefer unittest.mock and the stdlib.
• ALWAYS assert behaviour, not execution. For a tool wrapper, assert both the parsed result and the command that was built. Verify the patch target against the source (recon/tests/test_arjun.py broke when subprocess.run became Popen and the mocks kept targeting run).
• ALWAYS make a test that needs a stack, binary, service or git HEAD skip cleanly. A hard failure on a missing prerequisite is a bug in the test.