molly-calibration

Calibration

Triggered when a recruiter submits the "Review All Candidates" modal (via your Slack interactivity handler — see the README). Your prompt contains every decision at once: ROLE, CALIBRATION CYCLE, SPEC PAGE ID, JD PAGE ID, SLACK CHANNEL, SLACK THREAD TS, and a numbered list of candidates with APPROVED/REJECTED + optional feedback.

Steps

1. Update all rows in the calibration table

Read the spec page to find the calibration cycle table:

curl -s "https://api.notion.com/v1/blocks/SPEC_PAGE_ID/children?page_size=100" \
  -H "Authorization: Bearer $MOLLY_NOTION_API_KEY" \
  -H "Notion-Version: 2022-06-28"

Find the heading ## Calibration Cycle N (where N = CALIBRATION CYCLE). Get the table block that follows it. Get the table's row block IDs.

For each candidate, update the corresponding row (row index = candidate number, accounting for the header row at index 0):

curl -s -X PATCH "https://api.notion.com/v1/blocks/ROW_BLOCK_ID" \
  -H "Authorization: Bearer $MOLLY_NOTION_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Notion-Version: 2022-06-28" \
  -d '{"table_row": {"cells": [
    [{"type": "text", "text": {"content": "NAME", "link": {"url": "LINKEDIN_URL"}}}],
    [{"type": "text", "text": {"content": "WHY_IT_FITS"}}],
    [{"type": "text", "text": {"content": "yes_or_no"}}],
    [{"type": "text", "text": {"content": "FEEDBACK_OR_EMPTY"}}]
  ]}}'

Count approvals (APPROVED decisions). All rows will always have a result since the modal requires all decisions before submitting. Continue to step 2.

2. Analyze the cycle and write Calibration Notes

Read all rows. Count approvals (yes) and rejections (no). Identify patterns in the feedback.

Append ## Calibration Notes N to the spec page with a brief summary:

• How many approved / rejected
• What the feedback reveals about the spec (e.g. "Rejections suggest candidates with large audiences but without hands-on developer community management are a miss")
• Whether another calibration cycle is needed (yes if fewer than the full batch approved)