pyats-dynamic-test

Dynamic pyATS Test Execution

When to Use

• Automated pass/fail validation after configuration changes
• Compliance checks that require multi-step assertions
• Regression testing before and after maintenance windows
• Data-driven validation across multiple interfaces, neighbors, or routes
• Any scenario where you need a formal test report with PASSED/FAILED verdicts

How the Tool Works

The pyats_run_dynamic_test tool accepts a complete Python aetest script as a string. The script is executed in a sandboxed environment with a 300-second timeout.

PYATS_TESTBED_PATH=$PYATS_TESTBED_PATH python3 $MCP_CALL "${PYATS_PYTHON:-python3} -u $PYATS_MCP_SCRIPT" pyats_run_dynamic_test '{"test_script":"<FULL PYTHON SOURCE CODE>"}'

The test_script parameter takes the entire Python source code as a single JSON string. Newlines are encoded as \n, quotes as \".

Script Template Structure

Every aetest script follows this structure:

1. Imports (logging, aetest)
2. TEST_DATA dict (all expected values as a Python literal)
3. CommonSetup class (connect to devices)
4. Testcase class(es) (test methods with assertions)
5. CommonCleanup class (disconnect)
6. if __name__ == "__main__": aetest.main()

Rules for TEST_DATA

• Must be a Python dict literal defined at module level
• Contains all expected values the test will validate against
• Cannot be loaded from files, network, or environment variables
• Keeps the test deterministic and self-documenting

Rules for the Script

• Banned imports: os, sys, subprocess, shutil, socket, pathlib, pickle, yaml, requests, urllib, http, ssl
• Banned functions: __import__(), eval(), exec(), compile(), open(), json.loads()
• The script cannot connect to devices directly; embed all validation data inline in TEST_DATA
• Use logger.info() for verbose output and self.failed() or assert for test verdicts

Example 1: Interface State Validation

Verify that all expected interfaces are in up/up state.