03-consistency

03-consistency is an agent for coding agents from proggarapsody/bitbottle. It costs 0 tokens per session (3,616 once invoked), scanned A, original, MIT.

An analysis of why an automated process produced inconsistent results across a series of cycles, including its changing steps and recurring merge race.

In plain words
What is it for?
Use it to understand cycle results, compare periods, examine rework rates, and investigate the timing issue between checks and automatic merging.
Why use it?
It helps identify the causes of unstable outcomes and shows how the problem worsened over time.

Agent

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add agents/proggarapsody/bitbottle/03-consistency
Clone the repo
git clone --depth 1 https://github.com/proggarapsody/bitbottle

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for 03-consistency

README.md
[![agentmods](https://agentmods.dev/badge/agents/proggarapsody/bitbottle/03-consistency.svg)](https://agentmods.dev/agents/proggarapsody/bitbottle/03-consistency)
Your own site
<a href="https://agentmods.dev/agents/proggarapsody/bitbottle/03-consistency"><img src="https://agentmods.dev/badge/agents/proggarapsody/bitbottle/03-consistency.svg" alt="Measured on agentmods" height="20"></a>
Per session 0 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 3,616 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00000 $0.03616
Opus 5 $0.00000 $0.01808
Sonnet 5 $0.00000 $0.00723
Haiku 4.5 $0.00000 $0.00362

Measured 4d ago against content hash 9aa2dfa57da0, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

03-consistency scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

auto-iter/reports/analysis-2026-06-02-cycles-158-187/agents/03-consistency.md · 224 lines

How it starts

The opening of the file, as written. The whole thing — 224 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Agent 03 — Consistency & Predictability

Cycles 153–187 · Analysis date: 2026-06-02


TL;DR

The loop is not predictably consistent across cycles 153–187. Three distinct sub-eras exist with incompatible step schemas, missing logs, and sharply different rework rates. The single recurring root cause is the auto-merge race (CI ~2 min completes before DJ ~3–5 min; auto-merge fires; BLOCKER lands post-merge). This pattern appeared in 0 of 8 iteration cycles in era 153–161, 2 of 10 in era 168–177, and 5–6 of 9 in era 178–187 (56–67%). The trend is sharply worsening. Duration variance is high cross-era (CV 64%) but tight within each era. Pipeline version is superficially frozen at 2026.05.20 for all 35 cycles while the underlying step schema changed three times.


1. Outcome Stability

Sub-era breakdown (iteration cycles only, brainstorm excluded)

Sub-era Cycles (iteration) Shipped clean Shipped-with-fix Rework rate Trend
153–161 (May 25 stream) 7 7 0 0% — baseline
164 (isolated) 1 1 0 0%
168–177 (May 29–Jun 1 stream) 10 8 2 20% ↑ regression
178–187 (Jun 1 stream) 9 3–4 5–6 56–67% ↑↑ sharply worse

Overall 153–187 rework rate (iteration cycles): 7–8 of 27 = 26–30%.

Cycle 181 ambiguity

Cycle 181 (CLOUD-CODE-INSIGHTS) is logged as outcome=shipped in cycles.jsonl but the DJ metrics record blocker_count=1. The stream report classifies it as "shipped | clean." This implies the BLOCKER was caught and fixed pre-merge within the same cycle — the only cycle in 178–187 where the DJ-then-arm gate succeeded by timing luck. If counted as "shipped-with-fix" the era 178–187 rework rate is 6/9 = 67%; if excluded it is 5/9 = 56%.

Era 168–177 detail

Cycles 168–169 shipped but with dispatch violations (orchestrator worked inline rather than dispatching subagents). The violations do not show as shipped_with_fix in the log but represent a process failure. Post-correction (cycles 172–177), 6 of 6 were dispatch-clean. The 2 shipped_with_fix entries (176, 177) are both auto-merge races.

Read the full file on GitHub · 224 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 4d ago First seen · 224 lines · 0 tokens per session scan A 9aa2dfa57da0

Subscribe to this mod's changes

03-consistency is an agent published in the GitHub repository proggarapsody/bitbottle (3 stars, last pushed 16d ago), licensed MIT. It costs nothing until one of its globs matches a file; then it loads 3,616 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.