spacy

spaCy Best Practices

spaCy is the backbone of our NLP systems. Adhering to these guidelines ensures our pipelines are performant, reproducible, and easy to maintain.

1. Project Organization & Configuration

Always structure your spaCy projects using spacy project and define all pipeline settings in a declarative YAML config. This is non-negotiable for reproducibility and scalability.

✅ GOOD: Use spacy project and declarative configs. Organize your pipeline logic in a dedicated src/pipeline/ package. Use project.yml to manage workflows and config.cfg for all spaCy pipeline settings.

# src/pipeline/custom_component.py
from spacy.language import Language
from spacy.tokens import Doc

@Language.factory("my_custom_component")
def create_my_component(nlp: Language, name: str):
    return MyCustomComponent(nlp, name)

class MyCustomComponent:
    def __init__(self, nlp: Language, name: str):
        self.nlp = nlp
        self.name = name

    def __call__(self, doc: Doc) -> Doc:
        # Custom logic here
        return doc

# project.yml (simplified)
# ...
workflows:
  train:
    - "python -m spacy train config.cfg --output models/"
  package:
    - "python -m spacy package models/en_core_web_v1.0.0 ./dist --build wheel"

# config.cfg (simplified)
[nlp]
lang = "en"
pipeline = ["tok2vec", "ner", "my_custom_component"]

[components.my_custom_component]
factory = "my_custom_component"

❌ BAD: Ad-hoc scripts and hardcoded parameters. Avoid scattering pipeline logic across multiple scripts or hardcoding model paths and hyperparameters. This makes experiments non-reproducible and deployment fragile.

# bad_script.py
import spacy
# Parameters hardcoded or passed via CLI args, not in a central config
MODEL_PATH = "path/to/my/model"
THRESHOLD = 0.7

nlp = spacy.load(MODEL_PATH)
# ... pipeline components added programmatically ...

2. Type Hinting

Strictly use type hints for all spaCy objects (Language, Doc, Span, Token). This improves code readability, enables static analysis, and reduces runtime errors.