factor-evaluation

A tool for checking whether mathematical market signals, called factors, still work on data they were not developed on. It measures signal quality, trading activity, and the drop from training results to test results.

In plain words
What is it for?
Use it to recalculate factor metrics, compare training and test performance, measure turnover, and identify the strongest signals from a factor library.
Why use it?
It helps reveal overfitting: a factor may look good on the data used to create it but fail on new data.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/minihellboy/factorminer/factor-evaluation
Any agent
npx skills add minihellboy/factorminer --skill factor-evaluation
Clone the repo
git clone --depth 1 https://github.com/minihellboy/factorminer

Made for: Claude Code, Codex.

Per session 90 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 596 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00090 $0.00596
Opus 5 $0.00045 $0.00298
Sonnet 5 $0.00018 $0.00119
Haiku 4.5 $0.00009 $0.00060

Measured 2d ago against content hash f3be4331109c, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

factor-evaluation scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

integrations/factor-researcher/plugin/skills/factor-evaluation/SKILL.md · 56 lines

How it starts

The opening of the file, as written. The whole thing — 56 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Factor Evaluation

Mining proposes factors; evaluation decides whether to believe them. This skill recomputes a library's metrics on a chosen split and exposes overfitting.

See references/metrics.md for precise metric definitions (IC vs. paper-IC, ICIR, redundancy correlation).

Workflow

1. Recompute metrics

factorminer evaluate output/run1/factor_library.json \
  --data path/to/market_data.csv \
  --period test

--period selects the split: train, test, or both. Always lead with test — in-sample IC is not evidence.

2. Read the table

The output table reports, per factor: IC Mean, Paper IC, Abs IC, Paper ICIR, Win%, and Turnover. The summary block gives library-level means and the IC range.

3. Check decay

factorminer evaluate output/run1/factor_library.json --data market_data.csv --period both

--period both adds a decay table (train Paper IC → test Paper IC → delta). A large negative delta is the signature of an overfit factor. Report decay honestly; do not quote the train number as the headline.

4. Rank the survivors

To shortlist the strongest signals only:

factorminer evaluate output/run1/factor_library.json --data market_data.csv --period test --top-k 10

The top-K-by-IC table is the signal shortlist — the natural handoff to a research-idea workflow that wants to know which quantitative signals are currently working. The MCP screen_factors tool returns this same shortlist directly.

Interpreting the numbers

  • IC ≈ 0.03–0.05 out of sample is a respectable single factor on liquid universes.
  • ICIR matters more than IC: a small but stable IC beats a large erratic one.
  • High turnover quietly erases IC once costs are applied — carry it into factor-backtest.

Guardrails

  • Never present train metrics as the result. The deliverable is the test number.
  • If every factor decays to ~0 on test, the library failed — say so. Do not search for a split that flatters it.

Read the full file on GitHub · 56 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 56 lines · 90 tokens per session scan A f3be4331109c

Subscribe to this mod's changes

factor-evaluation is a skill published in the GitHub repository minihellboy/factorminer (105 stars, last pushed 16d ago), licensed MIT. It adds 90 tokens to every session and 596 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.