backtest-diagnose

Backtest Diagnosis

Overview

Use this skill when a user reports that a backtest failed, raised an error, or produced poor results.

Diagnostic Workflow

1. Read existing artifacts: use read_file to inspect artifacts/metrics.csv, equity.csv, and trades.csv
2. Read the code: use read_file to inspect code/signal_engine.py and config.json
3. Classify the issue: determine the root cause using the error taxonomy below
4. Apply the fix: use edit_file to modify the code, then rerun the backtest
5. Verify the fix: use read_file to inspect the new metrics.csv

Error Taxonomy

Runtime Errors (exit_code != 0)

Error Type	Common Cause	Fix
ImportError	Missing dependency	bash("pip install xxx")
KeyError	DataFrame column-name mismatch	Check the actual column names in data_map
IndexError	Empty data or insufficient length	Add length checks
TypeError	Incorrect signal type	Ensure the return value is pd.Series

Logic Bugs (Backtest Succeeds but Results Are Abnormal)

1. Zero trades (trade_count=0): signal-logic bug. Conditions are too strict, so the signal stays at 0. Check whether entry and exit logic is reasonable, and inspect the signal series to confirm it is not all zeros.
2. Late trades (first trade occurs more than 2 years after the backtest start): data-filtering bug. The lookback window may be too long, or the initial data segment may have been dropped. Shorten the window or check whether dropna is too aggressive.
3. Capital utilization < 50% (mostly in cash): position-sizing bug. Signal triggers may be too sparse, or the position-sizing logic may be wrong.
4. Open position at the end (a position still exists when the backtest ends): exit-timing bug. Forced liquidation may be missing, or exit logic does not cover the final segment.

Data Errors

Symptom	Root Cause	Fix
No data fetched	Invalid API token or code issue	Check config.json
Too little data	Date range too narrow	Expand the date range