automl-hyperparameter-optimization

A set of machine-learning guidelines for automatically comparing models and tuning their settings, known as hyperparameters.

In plain words
What is it for?
Use it when exploring models or tuning training settings with Python tools such as Ray Tune, Optuna, PyCaret, or time-series libraries.
Why use it?
It keeps automated experiments valid by defining baselines, preventing data leakage, recording results, and using suitable validation splits.

Cursor rule

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add rules/patrickjs/awesome-cursorrules/automl-hyperparameter-optimization
Clone the repo
git clone --depth 1 https://github.com/PatrickJS/awesome-cursorrules
Per session 0 Nothing until a file matches its globs; then the whole rule loads.
When invoked 577 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00000 $0.00577
Opus 5 $0.00000 $0.00289
Sonnet 5 $0.00000 $0.00115
Haiku 4.5 $0.00000 $0.00058

Measured 2d ago against content hash 3d683db3136a, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

automl-hyperparameter-optimization scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

rules/automl-hyperparameter-optimization.mdc · 56 lines

How it starts

The opening of the file, as written. The whole thing — 56 lines — stays where its author put it; the contents beside it link to each section on GitHub.

AutoML and Hyperparameter Optimization Rules

Scope

  • Use AutoML to accelerate model exploration, not to bypass problem framing, validation design, or explainability.
  • Start with a simple baseline model and fixed metric before launching a search.
  • Keep training, evaluation, feature generation, and search configuration separate.
  • Record datasets, splits, metric definitions, random seeds, library versions, and search spaces for every run.

Experiment Design

  • Define the target metric before selecting tooling.
  • Use nested validation or a final untouched test split for model selection claims.
  • Use time-aware splits for time-series problems; never shuffle across time boundaries.
  • Prevent leakage by fitting preprocessing only on training folds.
  • Include simple baselines such as linear models, random forests, or naive time-series forecasts.
  • Use early stopping and resource limits for expensive searches.
  • Prefer structured search spaces with domain-informed ranges over arbitrary broad grids.

Tooling

  • Use Ray Tune or Optuna for custom training loops, distributed trials, pruning, and scheduler control.
  • Use PyCaret for quick low-code comparisons when the dataset and metric are straightforward.
  • Use AutoTS, Merlion, PyAF, or project-approved time-series tooling when forecast-specific validation, seasonality, and horizon handling matter.
  • Store run metadata in MLflow, Weights & Biases, TensorBoard, or a project-approved tracker.
  • Use uv or the existing project package manager for reproducible environments.

Search Spaces

  • Keep search spaces explicit and reviewed.
  • Use log-scale sampling for learning rates, regularization, tree counts, and other scale-sensitive values.
  • Constrain model complexity to avoid unrealistic training time or memory use.
  • Include preprocessing choices only when they can be applied without leakage.
  • Do not tune on the test set.

Reporting

  • Report the selected model, metric, confidence interval or variance, validation scheme, and final test result.
  • Include the best parameters and the search budget.
  • Compare the chosen model against the baseline and at least one non-AutoML alternative.
  • Document operational constraints such as inference latency, memory use, retraining cost, and explainability.

Read the full file on GitHub · 56 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 56 lines · 0 tokens per session scan A 3d683db3136a

Subscribe to this mod's changes

automl-hyperparameter-optimization is a cursor rule published in the GitHub repository PatrickJS/awesome-cursorrules (40,694 stars, last pushed 3mo ago), licensed CC0-1.0. It costs nothing until one of its globs matches a file; then it loads 577 tokens. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.