arxiv-database

arXiv Database

Overview

This skill provides Python tools for searching and retrieving preprints from arXiv.org via its public Atom API. It supports keyword search, author search, category filtering, arXiv ID lookup, and PDF download. Results are returned as structured JSON with titles, abstracts, authors, categories, and links.

When to Use This Skill

Use this skill when:

• Searching for preprints in CS, ML, AI, physics, math, statistics, q-bio, q-fin, or economics
• Looking up specific papers by arXiv ID (e.g., 2309.10668)
• Tracking an author's recent preprints
• Filtering papers by arXiv category (e.g., cs.LG, cs.CL, stat.ML)
• Downloading PDFs for full-text analysis
• Building literature review datasets for AI/ML research
• Monitoring new submissions in a subfield

Consider alternatives when:

• Searching for biomedical literature specifically -> Use pubmed-database or biorxiv-database
• You need citation counts or impact metrics -> Use openalex-database
• You need peer-reviewed journal articles only -> Use pubmed-database

Core Search Capabilities

1. Keyword Search

Search for papers by keywords in titles, abstracts, or all fields.

python scripts/arxiv_search.py \
  --keywords "sparse autoencoders" "mechanistic interpretability" \
  --max-results 20 \
  --output results.json

With category filter:

python scripts/arxiv_search.py \
  --keywords "transformer" "attention mechanism" \
  --category cs.LG \
  --max-results 50 \
  --output transformer_papers.json

Search specific fields:

# Title only
python scripts/arxiv_search.py \
  --keywords "GRPO" \
  --search-field ti \
  --max-results 10

# Abstract only
python scripts/arxiv_search.py \
  --keywords "reward model" "RLHF" \
  --search-field abs \
  --max-results 30

2. Author Search

python scripts/arxiv_search.py \
  --author "Anthropic" \
  --max-results 50 \
  --output anthropic_papers.json

python scripts/arxiv_search.py \
  --author "Ilya Sutskever" \
  --category cs.LG \
  --max-results 20