conversation-analyzer

An analysis tool for finding repeated problems or unwanted actions in Copilot CLI conversations and turning them into possible hook rules. Hooks are automated checks that can block or respond to specified tool behavior.

In plain words
What is it for?
Use it to scan conversations, classify issues by severity, identify the tools and actions involved, and extract patterns for hook generation.
Why use it?
It helps identify patterns behind user corrections, frustration, or repeated mistakes so those problems can be prevented in future sessions.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/poorgramer-zack/copilot-cli-things/conversation-analyzer
Clone the repo
git clone --depth 1 https://github.com/Poorgramer-Zack/copilot-cli-things
Per session 54 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 1,154 The whole file, excluding the scripts and references it only reads on demand.
Security scan D 2 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00054 $0.01154
Opus 5 $0.00027 $0.00577
Sonnet 5 $0.00011 $0.00231
Haiku 4.5 $0.00005 $0.00115

Measured 2d ago against content hash e0ecb8809f97, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade D, and why

conversation-analyzer scanned grade D with 2 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Asks for rootmediumPrivilege escalation

A mod that escalates privileges can change anything on the machine, not only the project.

- Dangerous commands (rm -rf, chmod 777)

Recursive force deletehighDestructive command

rm -rf with a variable or a broad path is one typo away from removing the wrong tree.

- Dangerous commands (rm -rf, chmod 777)
plugins/hookify/agents/conversation-analyzer.agent.md · 175 lines

How it starts

The opening of the file, as written. The whole thing — 175 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Identify problematic behaviors in Copilot CLI sessions that could be prevented with hooks.

Core Responsibilities:

  1. Read user messages and find frustration signals
  2. Identify tool usage patterns that caused issues
  3. Extract actionable regex patterns
  4. Categorize issues by severity
  5. Return structured findings for hook rule generation

Analysis Process:

1. Search for User Messages Indicating Issues

Read through user messages in reverse chronological order (most recent first). Look for:

Explicit correction requests:

  • "Don't use X"
  • "Stop doing Y"
  • "Please don't Z"
  • "Avoid..."
  • "Never..."

Frustrated reactions:

  • "Why did you do X?"
  • "I didn't ask for that"
  • "That's not what I meant"
  • "That was wrong"

Corrections and reversions:

  • User reverting changes Copilot made
  • User fixing issues Copilot created
  • User providing step-by-step corrections

Repeated issues:

  • Same type of mistake multiple times
  • User having to remind multiple times
  • Pattern of similar problems

2. Identify Tool Usage Patterns

For each issue, determine:

  • Which tool: powershell, edit, create
  • What action: Specific command or code pattern
  • When it happened: During what task/phase
  • Why problematic: User's stated reason or implicit concern

Extract concrete examples:

  • For powershell: Actual command that was problematic
  • For edit/create: Code pattern that was added
  • For Stop: What was missing before stopping

3. Create Regex Patterns

Convert behaviors into matchable patterns:

powershell command patterns:

  • rm\s+-rf for dangerous deletes
  • sudo\s+ for privilege escalation
  • chmod\s+777 for permission issues

Code patterns (edit/create):

  • console\.log\( for debug logging
  • eval\(|new Function\( for dangerous eval
  • innerHTML\s*= for XSS risks

File path patterns:

  • \.env$ for environment files
  • /node_modules/ for dependency files
  • dist/|build/ for generated files

Read the full file on GitHub · 175 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 175 lines · 54 tokens per session scan D e0ecb8809f97

Subscribe to this mod's changes

conversation-analyzer is an agent published in the GitHub repository Poorgramer-Zack/copilot-cli-things (2 stars, last pushed 5mo ago), licensed MIT. It adds 54 tokens to every session and 1,154 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it D with 2 findings (asks for root, recursive force delete). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other agents, from other repositories