filter-log

Filter Log Command

Pick the smallest slice of a log that still answers the question you brought to it. Filter beats compression: on the committed intake_queue.jsonl fixture, tail -n 100 saves 95.6 percent of bytes while logs-tokenizer saves 70.3 percent. The asymmetry is reproducible and the tests/test_log_debugging_hygiene.py::test_filter_first_claim_is_reproducible test guards it.

This command routes the user to the tier-1 filters documented in skills/compression-strategy/modules/log-debugging-hygiene.md and stops there. Tier 3 (compression) is intentionally not the default path.

Usage

/conserve:filter-log path/to/file.log
/conserve:filter-log path/to/file.log --lines 50
/conserve:filter-log path/to/file.log --apply --tokens
/conserve:filter-log path/to/file.jsonl --apply

Arguments

Arg	Required	Meaning
<file>	yes	Path to the log file (.log, .jsonl, .txt, stdout dump).
--apply	no	Run the recommended filter and print the result. Default: print the command only.
--tokens	no	Measure pre- and post-filter token counts with tiktoken (cl100k_base proxy).
--lines N	no	Override the default line budget. Default: 100 for tail, 50 for head.

What This Command Does

When invoked, follow this routine. Do not skip steps.

Step 1: Inspect the file

Run these probes (read-only, fast):

wc -l "$FILE"          # total lines
wc -c "$FILE"          # total bytes
file "$FILE"           # type detection
head -n 3 "$FILE"      # first few lines for format
tail -n 3 "$FILE"      # last few lines for recency

Use the output to classify the log into one of four shapes:

• JSONL (each line parses as JSON) -> jq route
• Timestamped plaintext (lines start with ISO timestamp or [level]) -> rg or awk route
• Stack trace or panic -> rg -B 5 -A 20 route
• Unstructured -> tail or head route as fallback

Step 2: Recommend the tier-1 filter