A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
EvalScope is a framework for evaluating and stress-testing large language, vision-language, embedding, reranking, and generative models, with tools for running benchmarks and reviewing results. It is for developers and researchers comparing model capabilities, agent behavior, and inference performance. The catalogue add-ons provide workflows for running evaluations and interpreting their outputs.
This repository also configures its own agents. See what evalscope tells them →
Skill Claude CodeCodex
LLM evaluation & inference performance testing via the evalscope CLI. Translates natural language requests into evalscope commands for: (1) Model accuracy evaluation — runs registered benchmarks against local checkpoints or API endpoints (OpenAI-compatible, Anthropic, LiteLLM); (2) Performance stress testing — TTFT…