update-ai-docs

Update AI Docs

Autonomous pipeline: pull live rankings, detect changes, research new leaders, update docs. No human interaction.

Variables

target: $ARGUMENTS (optional — specific category to update, e.g., "code". If empty, checks all 10 categories for changes.)

Instructions

Run this pipeline end-to-end with zero stops, zero prompts, zero AskUserQuestion calls. If nothing changed, report that and exit. If something changed, research it and update the docs — all automatically.

Category Tiers

• Tier 1 (LMArena): text, code, vision, text-to-image, image-edit, search, text-to-video, image-to-video
• Tier 2: text-to-speech (TTS Arena V2), speech-to-text (Voice Writer Leaderboard)

Tier 1 uses ELO scores from arena.ai. Tier 2 uses ELO from TTS Arena (for TTS) and WER from Voice Writer (for STT, lower = better).

Phase 1: PULL LIVE RANKINGS

Tier 1 — LMArena (8 categories):

For each of the 8 LMArena categories, fetch the current leaderboard:

WebFetch https://arena.ai/leaderboard/{category}
Prompt: "Extract the complete leaderboard table. For each model return: rank, model name, ELO score (Arena Score), number of votes, organization/provider. Return the top 20 models as a structured list."

Tier 2 — TTS Arena + Voice Writer (2 categories):

For text-to-speech:

WebFetch https://tts-agi-tts-arena-v2.hf.space/leaderboard
Prompt: "Extract the TTS model leaderboard. For each model return: rank, model name, ELO score, win rate, total votes. Return all models."

For speech-to-text:

WebFetch https://voicewriter.io/speech-recognition-leaderboard
Prompt: "Extract the speech-to-text comparison table. For each system return: name, Mean WER, Std Dev, Price per hour. Return all models."

If a specific target category was given, only fetch that one.

Also query OpenRouter DB data for pricing context on text/code models:

SELECT model_name, price_input, price_output, context_length
FROM ai_models
WHERE source = 'openrouter' AND date = (SELECT MAX(date) FROM ai_models WHERE source = 'openrouter')
AND category IN ('text', 'code')
ORDER BY category, rank_in_category
LIMIT 30