tao-finetune-clip

tao-finetune-clip is a skill for Claude Code from NVIDIA-TAO/tao-skill-bank. It costs 104 tokens per session (3,844 once invoked), scanned A, original, Apache-2.0.

A training and deployment guide for CLIP, a model that connects images and text so software can compare or classify them.

In plain words
What is it for?
Use it for image-text retrieval, classifying images from text labels without task-specific training, extracting image embeddings, fine-tuning, ONNX export, and TensorRT deployment.
Why use it?
It provides task-specific guidance for adapting the model to specialised image-and-caption data and avoiding incorrect checkpoint or deployment assumptions.

Skill for Claude Code

Written for Claude Code: allowed-tools in frontmatter.

Part of the tao-skills plugin — 76 skills shipped together , and of tao-skill-bank

Good fit Use it for image-text retrieval, classifying images from text labels without task-specific training, extracting image embeddings, fine-tuning, ONNX export, and TensorRT deployment.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/nvidia-tao/tao-skill-bank/tao-finetune-clip
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add NVIDIA-TAO/tao-skill-bank --skill tao-finetune-clip
Clone the repo
git clone --depth 1 https://github.com/NVIDIA-TAO/tao-skill-bank

Made for: Claude Code.

Or install tao-skills, the plugin that ships this one along with the rest of its 76 skills.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for tao-finetune-clip

README.md
[![agentmods](https://agentmods.dev/badge/skills/nvidia-tao/tao-skill-bank/tao-finetune-clip/github.svg)](https://agentmods.dev/skills/nvidia-tao/tao-skill-bank/tao-finetune-clip)
Your own site
<a href="https://agentmods.dev/skills/nvidia-tao/tao-skill-bank/tao-finetune-clip"><img src="https://agentmods.dev/badge/skills/nvidia-tao/tao-skill-bank/tao-finetune-clip/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for tao-finetune-clip

Your own site · 80×15
<a href="https://agentmods.dev/skills/nvidia-tao/tao-skill-bank/tao-finetune-clip"><img src="https://agentmods.dev/badge/skills/nvidia-tao/tao-skill-bank/tao-finetune-clip.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 104 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 3,844 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • NVIDIA SkillSpector pass 7 Sept 2026
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00104 $0.03844
Opus 5 $0.00052 $0.01922
Sonnet 5 $0.00021 $0.00769
Haiku 4.5 $0.00010 $0.00384

Measured 12d ago against content hash a2ca51b27d97, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-12, from the pricing page.

Security

Grade A, and why

tao-finetune-clip scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/models/tao-finetune-clip/SKILL.md · 206 lines

How it starts

The opening of the file, as written. The whole thing — 206 lines — stays where its author put it; the contents beside it link to each section on GitHub.

CLIP

Standalone install? If this session was not initialized by the TAO skill bank plugin, run the tao-setup skill first (host preflight, credentials, cross-skill discovery).

Contrastive Language-Image Pre-training model for zero-shot and fine-tuned image classification, image-text retrieval, and embedding extraction. Fine-tuning adapts CLIP's shared image-text embedding space to domain-specific image-caption data.

No default NGC pretrained checkpoint is required for spec construction, but unset checkpoint behavior is action-specific. In the validation-fixes PyTorch image, export.checkpoint: null exports the selected CLIP architecture and may initialize weights when pretrained weights are unavailable. Do not assume inference.checkpoint: null loads pretrained weights: clip inference currently calls the checkpoint loader with None and fails before embedding extraction. For PyTorch inference, checkpoint-backed evaluation/export, resume, and retrain flows, resolve and pass an exact checkpoint from the parent train output. For trusted TAO checkpoints produced by the current run or a known parent job, set TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1 on checkpoint-dependent PyTorch actions so PyTorch 2.6 can load the Lightning checkpoint metadata; do not set this for untrusted checkpoints.

Supported actions: train, evaluate, inference, export, gen_trt_engine.

Train Action Policy

This model is AutoML-enabled at the model layer. Before handling any train-stage request, read references/skill_info.yaml and resolve the run override from either an explicit automl_policy value or the user's workflow request. Use automl_policy: on by default and only expose on / off in new launch prompts. Treat phrases like "turn off AutoML", "disable AutoML", "no HPO", or "plain training" as automl_policy: off for this run only. When automl_policy: on, automl_enabled: true, and both schemas/train.schema.json and references/spec_template_train.yaml are packaged, route the train action through tao-skill-bank:tao-run-automl by default with this model's skill_dir. Preserve workflow/application overrides for datasets, specs, output directories, GPU/platform settings, parent checkpoints, and automl_policy. Use direct model training only when automl_policy: off or the packaged train schema/template is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.

Read the full file on GitHub · 206 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 12d ago First seen · 206 lines · 104 tokens per session scan A a2ca51b27d97

Subscribe to this mod's changes

tao-finetune-clip is a skill published in the GitHub repository NVIDIA-TAO/tao-skill-bank (88 stars, last pushed today), licensed Apache-2.0. It adds 104 tokens to every session and 3,844 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

blip-2-vision-language

Vision-language pre-training framework bridging frozen image encoders and LLMs. Use when you need image captioning, visual question answering, image-text retrieval, or multimodal chat with state-of-the-art zero-shot performance.

davila7/claude-code-templates · 52 tokens

clip

OpenAI's model connecting vision and language. Enables zero-shot image classification, image-text matching, and cross-modal retrieval. Trained on 400M image-text pairs. Use for image search, content moderation, or vision-language tasks without fine-tuning. Best for general-purpose image understanding.

davila7/claude-code-templates · 60 tokens

yolo-master-agent

Use when the user wants to train, validate, predict, track, export, benchmark, tune, inspect, or orchestrate YOLO-Master / Ultralytics experiments in this repository, including LoRA, MoE, multimodal inference/evaluation, and solutions workflows.

Tencent/YOLO-Master · 60 tokens

blip-2-vision-language

Vision-language pre-training framework bridging frozen image encoders and LLMs. Use when you need image captioning, visual question answering, image-text retrieval, or multimodal chat with state-of-the-art zero-shot performance.

OpenLAIR/dr-claw · 52 tokens

clip

OpenAI's model connecting vision and language. Enables zero-shot image classification, image-text matching, and cross-modal retrieval. Trained on 400M image-text pairs. Use for image search, content moderation, or vision-language tasks without fine-tuning. Best for general-purpose image understanding.

OpenLAIR/dr-claw · 60 tokens

clip

OpenAI's model connecting vision and language. Enables zero-shot image classification, image-text matching, and cross-modal retrieval. Trained on 400M image-text pairs. Use for image search, content moderation, or vision-language tasks without fine-tuning. Best for general-purpose image understanding.

synthetic-sciences/openscience · 60 tokens