job-status

REINVENT4 Job Status

Monitor a running or completed REINVENT4 job by reading its log file and output CSV directly. No MCP server needed.

Workflow

1. Locate the log file

From $ARGUMENTS, determine the workdir or log path. The log is always at:

<workdir>/reinvent.log

If no argument provided, ask the user for the workdir or log path.

2. Check if the process is still running

If you know the PID:

kill -0 <PID> 2>/dev/null && echo "RUNNING" || echo "FINISHED"

Without PID, check if any reinvent process is active:

pgrep -la reinvent

3. Read the log

Quick status (last 20 lines):

tail -20 <workdir>/reinvent.log

For TL jobs, look for lines like:

• Epoch X/N — current epoch
• loss = — training loss
• Saving model — checkpoint written

For RL jobs, look for lines like:

• Step X or Iteration X — current step
• total_score = or Score: — current score
• Stage X completed — stage transition

4. Check output files

TL — list checkpoints and output model:

ls -lh <workdir>/*.model <workdir>/*.chkpt 2>/dev/null

RL — list stage CSVs and checkpoints:

ls -lh <workdir>/*.csv <workdir>/*.chkpt 2>/dev/null

5. For completed RL runs — summarise results

Read the stage CSV(s) with Python/pandas:

import pandas as pd, glob

csvs = sorted(glob.glob("<workdir>/*.csv"))
for csv in csvs:
    df = pd.read_csv(csv)
    print(f"\n{csv}")
    print(f"  Steps: {len(df)}")
    if "total_score" in df.columns:
        print(f"  Score — mean: {df['total_score'].mean():.3f}, max: {df['total_score'].max():.3f}")
    # show top 10 by score
    if "SMILES" in df.columns and "total_score" in df.columns:
        top = df.nlargest(10, "total_score")[["SMILES", "total_score"]]
        print(top.to_string(index=False))

6. Offer RL optimisation plots

For RL runs, offer to plot the training history. Ask which columns to include beyond the standard four (Agent NLL, Prior NLL, Augmented NLL, total_score):