transformers

Transformers

Overview

The Hugging Face Transformers library provides access to thousands of pre-trained models for tasks across NLP, computer vision, audio, and multimodal domains. Use this skill to load models, perform inference, and fine-tune on custom data.

Installation

Install transformers and core dependencies:

uv pip install torch transformers datasets evaluate accelerate

For vision tasks, add:

uv pip install timm pillow

For audio tasks, add:

uv pip install librosa soundfile

Credential Setup

HuggingFace token is auto-injected by openscience when connected via the dashboard.

# Verify credentials
[ -n "$HF_TOKEN" ] && echo "HF_TOKEN set" || echo "NOT SET"

If not set: add your Hugging Face token in Customize → Tools or export HF_TOKEN locally.

Quick Start

Use the Pipeline API for fast inference without manual configuration:

from transformers import pipeline

# Text generation
generator = pipeline("text-generation", model="gpt2")
result = generator("The future of AI is", max_length=50)

# Text classification
classifier = pipeline("text-classification")
result = classifier("This movie was excellent!")

# Question answering
qa = pipeline("question-answering")
result = qa(question="What is AI?", context="AI is artificial intelligence...")

Core Capabilities

1. Pipelines for Quick Inference

Use for simple, optimized inference across many tasks. Supports text generation, classification, NER, question answering, summarization, translation, image classification, object detection, audio classification, and more.

When to use: Quick prototyping, simple inference tasks, no custom preprocessing needed.

See references/pipelines.md for comprehensive task coverage and optimization.

2. Model Loading and Management

Load pre-trained models with fine-grained control over configuration, device placement, and precision.

When to use: Custom model initialization, advanced device management, model inspection.