distill-feedback

distill-feedback is a skill for Claude Code from AnastasiyaW/codex-claude-code-config. It costs 146 tokens per session (1,884 once invoked), scanned A, original, MIT.

A process for turning corrections you give an agent into lasting working rules. It reviews saved conversations and asks for approval before changing those rules.

In plain words
What is it for?
Use it to inspect queued feedback, identify durable lessons, and approve rules based on those lessons.
Why use it?
It reduces the need to repeat the same preferences or corrections in future sessions.

Skill for Claude Code

Written for Claude Code: SessionStart hook event. Also seen: reads .claude/ paths; mentions CLAUDE.md; mentions subagents.

Part of the claude-code-config plugin — 57 skills, 8 agents shipped together

Good fit Use it to inspect queued feedback, identify durable lessons, and approve rules based on those lessons.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/anastasiyaw/codex-claude-code-config/distill-feedback
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add AnastasiyaW/codex-claude-code-config --skill distill-feedback
Clone the repo
git clone --depth 1 https://github.com/AnastasiyaW/codex-claude-code-config

Made for: Claude Code.

Or install claude-code-config, the plugin that ships this one along with the rest of its 57 skills, 8 agents.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for distill-feedback

README.md
[![agentmods](https://agentmods.dev/badge/skills/anastasiyaw/codex-claude-code-config/distill-feedback/github.svg)](https://agentmods.dev/skills/anastasiyaw/codex-claude-code-config/distill-feedback)
Your own site
<a href="https://agentmods.dev/skills/anastasiyaw/codex-claude-code-config/distill-feedback"><img src="https://agentmods.dev/badge/skills/anastasiyaw/codex-claude-code-config/distill-feedback/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for distill-feedback

Your own site · 80×15
<a href="https://agentmods.dev/skills/anastasiyaw/codex-claude-code-config/distill-feedback"><img src="https://agentmods.dev/badge/skills/anastasiyaw/codex-claude-code-config/distill-feedback.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 146 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 1,884 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • NVIDIA SkillSpector warn 7 Sept 2026
SkillSpector: 2 findings, up to medium

These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →

  • medium Rogue Agent · line 44
    Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.
    Fix: Remove any persistence mechanisms (cron jobs, startup scripts, state files). Skills should not maintain state across sessions without explicit user consent.
  • medium Rogue Agent · line 44
    Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.
    Fix: Remove any persistence mechanisms (cron jobs, startup scripts, state files). Skills should not maintain state across sessions without explicit user consent.
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00146 $0.01884
Opus 5 $0.00073 $0.00942
Sonnet 5 $0.00029 $0.00377
Haiku 4.5 $0.00015 $0.00188

Measured yesterday against content hash 370e9daf86f6, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-08, from the pricing page.

Security

Grade A, and why

distill-feedback scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

The scan reads SKILL.md. This mod also ships 1 executable file (scripts/extract_feedback_queue.py), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/development/distill-feedback/SKILL.md · 116 lines

How it starts

The opening of the file, as written. The whole thing — 116 lines — stays where its author put it; the contents beside it link to each section on GitHub.

distill-feedback — close the learn-from-corrections loop

The Stop hook session-feedback-capture.py queues finished sessions into ~/.claude/feedback/queue.jsonl. This skill processes that queue: it finds the user turns that were durable corrections of the agent's work and turns them into rules — so the same correction never has to be given twice.

Why semantic review: standing preferences depend on context, not a trigger-word count. The formerly cited private effectiveness-test/RESULTS.md was absent when checked on 2026-09-06; its F1 claims are withdrawn from this skill until the dataset, labels, model, held-out split and raw predictions can be inspected. The rubric below is a review method, not a demonstrated accuracy guarantee. Do not substitute another paper's scores for our own.

Research and authority: ACE studies evolving context; TRACE studies compiling corrections into runtime checks. Neither proves this local extractor's accuracy. The approval boundary comes from our applicable user instructions and autonomy-risk-tiers.md, not an inferred paper mandate. New standing rules are proposals; an already authorized correction to an existing rule can be implemented within that exact authority. Codex memory changes also require the separate explicit user request and supported memory-update channel.

Procedure

1. Extract the queue (deterministic)

python ~/.claude/skills/distill-feedback/scripts/extract_feedback_queue.py --limit 8

Returns JSON: {pending, sessions:[{session_id, cwd, ts, user_turns:[...]}]}. --limit bounds the LLM pass (billing: distillation is opt-in, not every-session). If pending is 0, stop — nothing to do.

2. Detect durable corrections (LLM-semantic, prefer a fresh sub-agent)

For independence (Generator-Evaluator), spawn a sub-agent with the rubric below and the extracted user_turns. Ask it to return, per genuine correction: {quote, durable_rule, applicability_condition, confidence, session_id}. Pass only the turns — not your own reasoning.

Read the full file on GitHub · 116 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday Changed · +14 lines 370e9daf86f6
  2. 9d ago First seen · 102 lines · 146 tokens per session scan A 99bf0b68c9ed

Subscribe to this mod's changes

distill-feedback is a skill published in the GitHub repository AnastasiyaW/codex-claude-code-config (149 stars, last pushed today), licensed MIT. It adds 146 tokens to every session and 1,884 once invoked, about $0.0007 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

screenpipe-chats

Search what was said in existing screenpipe, Codex, Claude, Cursor, and Gemini CLI chats, then continue or steer one exact chat when the user explicitly asks. Use for requests about finding another agent conversation, recalling what an agent said, or sending work to one.

screenpipe/screenpipe · 60 tokens

knowledge-base

Retrieves and updates project-specific prompt knowledge from comparison evidence and user feedback. Use only for prompt analysis or post-comparison learning within a Rashomon evaluation.

shinpr/rashomon · 34 tokens

compress

Prompt-Studio memory compressor. Shrinks CLAUDE.md, todo lists, and preference files into Prompt-Studio's lean format to cut input tokens on every load. Preserves all technical substance, code, URLs, and structure. Compressed output overwrites the original; human-readable backup stored out-of-tree as FILE.original.md…

utk2103/Prompt-Studio · 98 tokens

add-new-entry-from-temp-md

Workflow and tools for adding new entries from temp.md to the section files. Includes legend format, section reference, code tools, and common pitfalls. USE FOR: Adding new resources to the knowledge base. DO NOT USE FOR: Editing existing entries or restructuring sections.

kimtth/azure-openai-llm-notes · 59 tokens

a-memory-is-not-a-trigger

Use when you're relying on a stored memory, a recalled note, an injected session-start banner, or a description-matched skill to make a required action happen every session or run — especially a recurring logging, bookkeeping, or guardrail step. Passive context is advisory: the agent reads it and can still skip the…

iamakbarsha1/whetstone · 163 tokens

tldr-compress

Compress natural language memory files (CLAUDE.md, todos, preferences) into TLDR format to save input tokens. Preserves all technical substance, code, URLs, and structure. Compressed version overwrites the original file. Human-readable backup saved as FILE.original.md. Trigger: /tldr-compress FILEPATH or "compress…

0p9b/TLDR · 75 tokens