Diffusers is a PyTorch library for running and building diffusion systems that generate images, audio, and 3D molecular structures. Developers use its pretrained models, generation pipelines, and interchangeable noise schedulers for inference or model training. The catalogue add-ons support workflows around the library.
Borrowing it
Nothing to install: this file belongs to huggingface/diffusers. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/huggingface/diffusers/main/.ai/skills/model-integration/SKILL.mdgit clone --depth 1 https://github.com/huggingface/diffusersWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/huggingface/diffusers/model-integration)<a href="https://agentmods.dev/skills/huggingface/diffusers/model-integration"><img src="https://agentmods.dev/badge/skills/huggingface/diffusers/model-integration/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/huggingface/diffusers/model-integration"><img src="https://agentmods.dev/badge/skills/huggingface/diffusers/model-integration.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- Snyk pass
- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00045 | $0.01861 |
| Opus 5.5 | $0.00018 | $0.00744 |
| Sonnet 5.5 | $0.00009 | $0.00372 |
| Haiku 4.5 | $0.00005 | $0.00186 |
Grade A, and why
model-integration scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 124 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Goal
Integrate a new model into diffusers end-to-end, to full numerical parity with the reference implementation — one workflow at a time.
Setup — gather before starting
Before writing any code, gather info in this order:
- Reference repo — ask for the github link. If they've already set it up locally, ask for the path. Otherwise, ask what setup steps are needed (install deps, download checkpoints, set env vars, etc.) and run through them before proceeding.
- Inference script — ask for a runnable end-to-end script for a basic workflow first (e.g. T2V). Then ask what other workflows they want to support (I2V, V2V, etc.) and agree on the full implementation order together.
- Standard vs modular — default to modular. Modular Diffusers is the preferred implementation for new pipelines; the standard
DiffusionPipelineis still supported but no longer the default. We prefer modular especially for models that don't fit a fixed task-based structure (modality baked into the checkpoint) or that are actively evolving.
Ask step 3 as an AskUserQuestion, with modular marked as the recommended default.
Once you have everything, confirm the plan with the user before implementing — state exactly what you'll do, e.g. "I'll integrate model X with pipeline Y based on your script, and verify the model matches the reference before considering it done."
Then work through the Integration checklist below
Integration checklist
A pipeline in Diffusers (be it standard or modular) will have multiple components. These components can be models, schedulers, processors, etc.
- Transformer model
- Implement the model with
from_pretrainedsupport (conventions: models.md) - Convert weights (see Weight / Checkpoint Conversion)
- Parity test against the reference (internal, not shipped — see Model parity test)
- Register in the relevant
__init__.pyfiles (lazy imports) - Model-level tests (see Testing)
- Implement the model with
- VAE (if applicable) — reuse an existing
AutoencoderKL*if possible; if a new one is needed, follow the same sub-steps as the transformer - Scheduler — reuse an existing scheduler, or add a custom one
- Pipeline
- Implement the pipeline — see modular.md for modular pipeline, or pipelines.md for standard pipeline
- Add a LoRA mixin if applicable
- Register in the relevant
__init__.pyfiles (lazy imports) - Pipeline-level tests (see Testing)
- Docs — see File structure
- Style —
make styleandmake quality
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 124 lines · 45 tokens per session scan A 16faeacecc0b
model-integration is a skill published in the GitHub repository huggingface/diffusers (34,728 stars, last pushed today), licensed Apache-2.0. It adds 45 tokens to every session and 1,861 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
ml-training-recipes
Battle-tested PyTorch training recipes for all domains — LLMs, vision, diffusion, medical imaging, protein/drug discovery, spatial omics, genomics. Covers training loops, optimizer selection (AdamW, Muon), LR scheduling, mixed precision, debugging, and systematic experimentation. Use when training or fine-tuning…
physicsnemo-shard-tensor
Official NVIDIA-authored guidance for PhysicsNeMo ShardTensor domain parallelism — integrate domain parallelism into training/inference scripts (new or existing) with DDP or FSDP2, write and register shard patches to enable new layers/ops, and bootstrap multi-GPU correctness tests. Use when working with ShardTensor…
physicsnemo-discover
Official NVIDIA-authored guidance for navigating PhysicsNeMo — pick the model, datapipe, or example for a SciML/AI4Science task (surrogates, forecasting, downscaling, physics-informed, inverse, generative). Points at existing files via live repo search; never writes code. Do NOT use for installation or environment…
deep-learning
Use when training or debugging a neural net in PyTorch — the forward/loss/backward/step loop and its silent bugs, mixed precision (AMP), AdamW/LR schedules, DDP/FSDP/ZeRO, checkpoints and seeds. NOT LoRA/QLoRA on a pretrained LLM (that is finetuning), NOT tabular sklearn/XGBoost (that is machine-learning), NOT…
single-cell-multi-omics-analysis-scvi
Probabilistic deep learning framework for single-cell multi-omics data analysis. Use this skill when: (1) Analyzing single-cell RNA-seq data with batch correction, (2) Integrating multi-modal data (CITE-seq, ATAC-seq, multi-omics), (3) Performing cell type annotation with scANVI, (4) Spatial transcriptomics…
ml-for-aec
Computer vision for buildings, image-to-floorplan, generative ML models, performance prediction, structural analysis ML, energy prediction, natural language to design, and point cloud ML for AEC computational design.