Gym: Skill for Codex

Nemo Gym Pivot Datasets

Paper Reference

This skill operationalizes PivotRL: create local single-step pivot datasets from successful trajectories, prefer informative mixed-reward states, and train with verifier-based local rewards rather than exact trajectory imitation.

Invocation Check

Use this skill when the task is to turn existing agent trajectories or rollout artifacts into a Nemo Gym pivot dataset, or to validate whether a pivot JSONL/config pair can be used for single-step local RL or evaluation.

Before writing a converter, inspect representative source rows and the target resource server. Do not assume the source field names are the contract. Convert by reconstructing the semantic pieces needed by Gym's Responses-style row format.

Core Workflow

1. Inspect the source data shape and count the candidate assistant decision points.
2. Reconstruct model calls from the source's flattened output list, and cross-check the count against whatever per-trajectory model-call count the source records. A decision point is one whole model call, so this step decides what a row even is.
3. Identify the semantic fields needed for each pivot:

• model-call input context before the pivot action
• available tools at that decision point
• expected assistant action
• reward/verifier target if it is separate from the demonstrated action
• optional provenance such as task id, source trajectory id, rollout id, uuid, depth, and original metadata

1. Convert each accepted decision point into one pivot JSONL row.
2. Generate or update the matching Gym config so the pivot-format JSONL can be used directly.
3. Validate with the bundled validator and, when available, the target Gym resource-server models.
4. Write metrics that make skipped rows, action types, tool names, depth, and provenance coverage easy to inspect, including the parallel / single / chat split and a batch-size histogram.

Row Shape

Read references/row-contract.md when implementing or reviewing a converter. For single_step_tool_use_with_argument_comparison, the essential row fields are: