transformers

Transformers: Pre-trained Language Models

Overview

Hugging Face Transformers provides thousands of pre-trained models for various modalities including text, image, audio, and video. Apply this skill for NLP tasks (text classification, NER, QA, generation), transformer-based models, fine-tuning, and model inference.

When to Use This Skill

This skill should be used when:

• Working with pre-trained transformer models (BERT, GPT, Llama, T5, etc.)
• Performing text classification or sentiment analysis
• Named Entity Recognition (NER)
• Question answering
• Text generation and completion
• Machine translation
• Text summarization
• Fine-tuning transformers on custom datasets
• Using tokenizer and model pipelines
• Loading models from Hugging Face Hub

Quick Start

Basic Import and Setup

from transformers import AutoModel, AutoTokenizer, pipeline
import torch

# Check device
device = 0 if torch.cuda.is_available() else -1
print(f"Using device: {'GPU' if device >= 0 else 'CPU'}")

Loading Pre-trained Models

# Load model and tokenizer
model_name = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

# Move to GPU if available
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

Using Pipelines

# Text classification
classifier = pipeline("text-classification", model="distilbert-base-uncased-finetuned-sst-2-english")
result = classifier("I love this product!")
print(result)

# Named Entity Recognition
ner = pipeline("ner", model="dbmdz/bert-large-cased-finetuned-conll03-english", aggregation_strategy="simple")
result = ner("Hugging Face is based in New York City.")
print(result)

# Question Answering
qa = pipeline("question-answering", model="distilbert-base-uncased-distilled-squad")
result = qa(question="What is Hugging Face?", context="Hugging Face is a company specializing in NLP.")
print(result)

# Text Generation
generator = pipeline("text-generation", model="gpt2")
result = generator("Once upon a time", max_length=50, num_return_sequences=1)
print(result[0]['generated_text'])

# Summarization
summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
result = summarizer("The article to summarize goes here...")
print(result)

# Translation
translator = pipeline("translation_en_to_fr", model="t5-small")
result = translator("Hello, how are you?")
print(result)