refine

/refine -- RLM SelfRefine Pipeline

Step 0: Preamble

Before executing, run the shared ProductionOS preamble (templates/PREAMBLE.md).

You are the RLM Refine orchestrator. You process pending signals from the RLM classifier, showing the user flagged outputs and applying L17 SelfRefine to improve them.

Input

• Mode: $ARGUMENTS.mode (default: interactive)
• Max signals: $ARGUMENTS.max_signals (default: 10)

Step 1: Load Pending Signals

Read pending signals from ~/.productionos/recursive/pending/:

python3 -c "
import json, os
from pathlib import Path
pending_dir = Path(os.path.expanduser('~/.productionos/recursive/pending'))
if not pending_dir.exists():
    print(json.dumps({'signals': [], 'count': 0}))
else:
    signals = []
    for f in sorted(pending_dir.glob('*.json')):
        try:
            data = json.loads(f.read_text())
            if not data.get('reviewed', False):
                data['_path'] = str(f)
                signals.append(data)
        except: pass
    print(json.dumps({'signals': signals, 'count': len(signals)}, indent=2, default=str))
"

If no unreviewed signals exist, report: "No pending signals to review. All outputs have been reviewed." and exit.

Step 2: Display Signal Summary

For each pending signal, display:

SIGNAL {id} [{verdict}] — Tool: {tool_name} — Score: {score:.2f}
  Agent: {agent_id or "session"}
  Time: {formatted_timestamp}
  Dimensions:
    - {dimension_name}: {score} — {reason}
    - ...

Group signals by verdict: show BLOCKs first (highest priority), then FLAGs.

Step 3: Process Each Signal

For BLOCK signals:

BLOCKed outputs indicate serious quality issues. In interactive mode:

1. Display the full signal details including all dimension scores
2. Ask the user: "This output was BLOCKED. Options: [A]ccept anyway, [R]eject, [S]kip"
3. Based on response:
   • Accept: Mark as reviewed: true, human_verdict: "accept", log to metrics
   • Reject: Mark as reviewed: true, human_verdict: "reject", log to metrics
   • Skip: Leave unreviewed for next session