sumo-qa-finding-test-data

Finding test data

Announce at start: "Finding test data — validated fresh."

Output discipline (mandatory)

Inherits the global discipline from using-sumo-qa: output discipline (never surface internal taxonomy labels — say "behaviour change in pricing", not "Classification: business_logic_change"), output economy (spend output on findings not framing; no preamble or self-narration; one question per turn; no closing pleasantries), knowledge authority hierarchy, internal scaffolding stays internal, and specialty-tool fit.

The Iron Law

STALE IS A DEFECT. NEVER INVENT ENTRIES NOT IN THE CATALOGUE.

A catalogued entry not re-validated against the source system in this turn is no better than an invented one — validation is the proof, not ceremony.

find_test_data MUST be immediately followed by validate_test_data against the returned record IN THE SAME TURN before the record is considered usable.

When to Use

User intents that trigger this skill:

• "what test data do I need for X"
• "find me a known-good record for X"
• "is this record / account / fixture still valid"
• "save this as known-good test data"
• "I need a locked account for the failed-login test"
• "I need a pending invoice for the due-date boundary test"

Checklist

Track these as an ordered work list (use the host's task primitive if available, otherwise a numbered inline tracker) and complete in order. Steps 1–2 are AI-only homework (route the request, gather inputs from prior conversation). The user's confirmation gate only applies to register (step 5b) — writing to the shared catalogue is a side effect that should always pause for confirmation.

1. Pick the route (no user question — derive from intent). The four routes are internal dispatch data, NOT a label to echo at the user:
   • Explain requirements: "what data do I need" → sumo_qa_explain_test_data_requirements
   • Find: "find me a record" → sumo_qa_find_test_data
   • Validate: "is this still valid" → sumo_qa_validate_test_data
   • Register: "save this as known-good" → sumo_qa_register_known_good_test_data
2. Gather inputs from intent + prior conversation (no user question if the conversation has them). Question, environment, domain, criteria, entry path — pull from what's already been said. Only ask if a genuinely required field is missing, and ask ONCE.
3. For explain: call the tool. Return the requirements as text in natural English.
4. For find: call the tool with the gathered criteria. If matches: pipe each into validate (step 5a). If no matches: say so explicitly — do NOT invent and do NOT silently broaden the criteria.
5. For validate:
   • 5a (from find or direct): call sumo_qa_validate_test_data against the source system in this turn. A cached or memorised result is not fresh. Report state with timestamp + the validation evidence.
   • 5b (register): confirm with the user before writing to the catalogue. Show the entry, the validation evidence, and ask: "Register this as known-good?" Only write on explicit yes. Arg shape: call sumo_qa_register_known_good_test_data(entry={...}) with entry as a literal dict (id, environment, domain, scenario_tags, known_valid_for, constraints, owner, last_validated_at, confidence, source, notes). Not a YAML string. The tool docstring has a complete example.
6. If an entry fails validation: mark it stale, surface the failure to the user, do NOT silently substitute another entry. The user decides whether to find an alternative or register a new one.