huggingface-hub

Hugging Face Hub: Model & Dataset Management

Overview

Hugging Face Hub is the platform for sharing machine learning models, datasets, and demos. Apply this skill for downloading/uploading models, browsing the hub, managing repositories, version control, and accessing pre-trained assets.

When to Use This Skill

This skill should be used when:

• Downloading pre-trained models from Hugging Face Hub
• Uploading trained models to share
• Browsing and searching for models
• Reading model cards and documentation
• Managing model versions
• Downloading datasets from the hub
• Creating and managing model repositories
• Using Git-based version control for models
• Setting up model inference endpoints

Quick Start

Basic Import and Setup

from huggingface_hub import HfApi, hf_hub_download, list_models, list_datasets
import os

# Authentication (optional for public models)
# Set token in environment: HF_TOKEN or use login()
from huggingface_hub import login
login(token="your_token_here")

Downloading Models

from huggingface_hub import hf_hub_download

# Download specific file
model_path = hf_hub_download(
    repo_id="bert-base-uncased",
    filename="pytorch_model.bin",
    # revision="main"  # branch name or commit hash
)

# Download config
config_path = hf_hub_download(
    repo_id="bert-base-uncased",
    filename="config.json"
)

# Download to specific directory
model_path = hf_hub_download(
    repo_id="meta-llama/Llama-2-7b-hf",
    filename="model-00001-of-00002.safetensors",
    local_dir="./models/llama-2-7b",
    local_dir_use_symlinks=False
)

Using with Transformers

from transformers import AutoModel, AutoTokenizer

# Load directly from hub (most common)
model = AutoModel.from_pretrained("bert-base-uncased")
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

# Load from specific revision
model = AutoModel.from_pretrained("bert-base-uncased", revision="v1.0.0")

# Load with specific cache directory
model = AutoModel.from_pretrained("bert-base-uncased", cache_dir="./cache")