esm2

ESM2: Evolutionary Scale Modeling

Overview

ESM2 is a family of encoder-only protein language models from Meta FAIR, trained on 250M+ UniRef50 sequences. Unlike ESM3 (which is generative), ESM2 is discriminative — it cannot generate sequences but excels at:

• Protein embeddings — dense representations for downstream ML tasks
• Zero-shot variant effect scoring — predict mutational fitness without labels
• Contact prediction — residue-residue contact maps from attention heads
• Structure prediction — via ESMFold (uses ESM2 as backbone)

ESM2 is available on HuggingFace with no special SDK or API token required. The same HF API also covers ESM-1b and ESM-1v (variant-specialized) — use AutoTokenizer / AutoModel for any of them interchangeably.

Installation

pip install transformers torch

For faster inference with GPU:

pip install transformers torch accelerate

To use the fair-esm package directly (alternative, gives access to ESM-1v and ESM-IF1):

pip install fair-esm

Model Selection

HuggingFace model ID	Params	Layers	Hidden dim	Use case
facebook/esm2_t6_8M_UR50D	8M	6	320	CPU, fast prototyping
facebook/esm2_t12_35M_UR50D	35M	12	480	CPU-friendly, good quality
facebook/esm2_t30_150M_UR50D	150M	30	640	Balanced
facebook/esm2_t33_650M_UR50D	650M	33	1280	Best default — GPU recommended
facebook/esm2_t36_3B_UR50D	3B	36	2560	High accuracy, needs GPU
facebook/esm2_t48_15B_UR50D	15B	48	5120	Max accuracy, multi-GPU

Start with esm2_t33_650M_UR50D unless compute is constrained.

Core Capabilities

1. Protein Embeddings

Extract per-residue or per-sequence embeddings for downstream tasks (classification, clustering, regression).

from transformers import EsmTokenizer, EsmModel
import torch

model_name = "facebook/esm2_t33_650M_UR50D"
tokenizer = EsmTokenizer.from_pretrained(model_name)
model = EsmModel.from_pretrained(model_name).eval()

sequence = "MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPILSRVGDGTQDNLSGAEKAVQVKVKALPDAQFEVVHSLAKWKRQTLGQHDFSAGEGLYTHMKALRPDEDRLSPLHSVYVDQWDWERVMGDGERQFSTLKSTVEAIWAGIKATEAAVSEEFGLAPFLPDQIHFVHSQELLSRYPDLDAKGRERAIAKDLGAVFLVGIGGKLSDGHRHDVRAPDYDDWSTPSELGHAGLNGDILVWNPVLEDAFELSSMGIRVDADTLKHQLALTGDED"

inputs = tokenizer(sequence, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

# Per-residue embeddings: shape (seq_len, hidden_dim)
# Slice [1:-1] to remove [CLS] and [EOS] special tokens
per_residue = outputs.last_hidden_state[0, 1:-1]

# Per-sequence embedding: mean pool over residues
per_sequence = per_residue.mean(dim=0)  # shape (hidden_dim,)

print(f"Per-residue: {per_residue.shape}")   # (L, 1280)
print(f"Per-sequence: {per_sequence.shape}") # (1280,)