prepare-release

Prepare a Release

This workflow prepares a new release of inspect_evals. It creates a release branch, collects changelog fragments, bumps the version tag, and opens a PR. After merge it tags the merge commit and creates a GitHub release.

Reference: PACKAGE_VERSIONING.md

Prerequisites

• The gh CLI must be authenticated
• You must have push access to the repository
• There must be changelog fragments in changelog.d/ (beyond .gitkeep and TEMPLATE.md)

Phase 1 — Prepare the release branch

1. Ensure main is up to date:

   git fetch origin
   git checkout main
   git merge --ff-only origin/main
   

   If the merge fails, stop and inform the user that their local main has diverged from origin/main.

2. Check for changelog fragments:

   List files in changelog.d/ excluding .gitkeep, TEMPLATE.md, and README.*. If there are no fragments, stop and tell the user there is nothing to release.

3. Determine the new version:

   1. Find the current version from the latest v* git tag:

      git tag --sort=-v:refname --list 'v*' | head -1
      

   2. Ask the user what kind of bump this is, presenting the semver table from PACKAGE_VERSIONING.md:

      Component	When to bump	Examples
      Major	Breaking changes	Removing an eval, API changes, scorer output format changes
      Minor	New features	Adding new evals, new task parameters, new utilities
      Patch	Bug fixes	Eval fixes, scorer fixes, dataset loading fixes

   3. Compute the new version string (e.g. 0.4.0, 0.3.107). Confirm with the user.

4. Create the release branch:

   The branch name is release-YYYY-MM-DD using today's date.

   git checkout -b release-YYYY-MM-DD
   

Phase 2 — Collect the changelog