policy-optimization

Policy Optimization

Overview

Use this skill when /policy output shows drift, stalls, or noisy failure patterns. Goal: improve promotion quality and stability without unsafe threshold changes.

Core principle: diagnose first, tune second. Do not change thresholds without clear evidence from attempts, success_rate, verify_rate, and failure patterns.

When to Use

• explore count is high and long-lived.
• Any pipeline has consecutive_failures >= 1.
• canary pipelines fail to reach stable despite enough attempts.
• Rollbacks are frequent (rollback_executed_count grows).
• The team asks "which policy threshold should we tune next?"

Do not use when:

• The request is only to display status (use policy command only).
• There are no meaningful signals (too little data / low attempts).

Workflow

1) Capture policy snapshot

python3 "${CLAUDE_PLUGIN_ROOT}/scripts/repl_admin.py" policy-status --pretty

If parsing is needed:

python3 "${CLAUDE_PLUGIN_ROOT}/scripts/repl_admin.py" policy-status

2) Classify risk buckets

Classify each pipeline into one bucket:

• Critical: consecutive_failures >= rollback_consecutive_failures
• Warning: consecutive_failures == 1 or verify_rate materially low
• Stalled: high attempts but still explore/canary
• Healthy: stable or trend strongly positive

3) Prioritize remediation

Priority order:

1. Fix Critical pipelines first (execution correctness and rollback safety)
2. Fix high-volume Warning pipelines
3. Promote Stalled but healthy candidates (remove lifecycle friction)
4. Leave Healthy unchanged

Tie-breakers:

• Higher consecutive_failures first
• Then lower verify_rate
• Then higher policy traffic (policy_enforced_count)

4) Propose threshold tuning (guardrailed)

Threshold changes are allowed only when:

• Sample size is credible (attempts near/above promotion thresholds)
• Signal is consistent across multiple pipelines (not one-off noise)
• A specific failure mode is identified