This skill should be used when the user wants to evaluate an AI agent end-to-end: scaffold an evaluation, design metrics that test a real hypothesis, make an agent measurable, audit generated eval config, read evaluation results, or run an improvement ("hill climbing") loop. Covers evaluation methodology, metric…
Executes high-performance agent evaluations, multi-turn UserSim simulations, and declarative metric grading aligned with google/agents-cli and the Quality Flywheel. Publishes benchmark artifacts to the GCS Evaluation Registry, executes automated head-to-head delta comparisons (--compare-to), and optimizes system…
Performs an exhaustive, question-by-question narrative diagnostic breakdown of an agent-eval benchmark run by analyzing questionanswerlog.md, evalsummary.json, and raw trajectory traces. Use when diagnosing low score causes, investigating the Memory Reuse vs. Traceability rubric clash, performing pre-release failure…
Guides the usage of Gemini API on Google Cloud Vertex AI with the Gen AI SDK. Use when the user asks about using Gemini in an enterprise environment or explicitly mentions Vertex AI. Covers SDK usage (Python, JS/TS, Go, Java, C#), capabilities like Live API, tools, multimedia generation, caching, and batch prediction.
Skill "vertex-deploy" from GoogleCloudPlatform/vertex-ai-samples, covering vertex ai model garden deploy skill, 1. prerequisites, 2. discovering deployable models, 3. deploying a model and example: deploying gemma 3.
Skill "vertex-inference" from GoogleCloudPlatform/vertex-ai-samples, covering vertex ai genai inference skill, 1. authentication (critical), 2. gemini models, choosing the right sdk and installation.