transformers

Transformers

Overview

The Hugging Face Transformers library provides access to thousands of pre-trained models for tasks across NLP, computer vision, audio, and multimodal domains. Use this skill to load models, perform inference, and fine-tune on custom data.

Installation

Install transformers and core dependencies:

uv pip install torch transformers datasets evaluate accelerate

For vision tasks, add:

uv pip install timm pillow

For audio tasks, add:

uv pip install librosa soundfile

Authentication

Many models on the Hugging Face Hub require authentication. Set up access:

from huggingface_hub import login
login()  # Follow prompts to enter token

Or set environment variable:

export HUGGINGFACE_TOKEN="your_token_here"

Get tokens at: https://huggingface.co/settings/tokens

Quick Start

Use the Pipeline API for fast inference without manual configuration:

from transformers import pipeline

# Text generation
generator = pipeline("text-generation", model="gpt2")
result = generator("The future of AI is", max_length=50)

# Text classification
classifier = pipeline("text-classification")
result = classifier("This movie was excellent!")

# Question answering
qa = pipeline("question-answering")
result = qa(question="What is AI?", context="AI is artificial intelligence...")

Core Capabilities

1. Pipelines for Quick Inference

Use for simple, optimized inference across many tasks. Supports text generation, classification, NER, question answering, summarization, translation, image classification, object detection, audio classification, and more.

When to use: Quick prototyping, simple inference tasks, no custom preprocessing needed.

See references/pipelines.md for comprehensive task coverage and optimization.

2. Model Loading and Management

Load pre-trained models with fine-grained control over configuration, device placement, and precision.

When to use: Custom model initialization, advanced device management, model inspection.