agentic-paper-digest-skill

Agentic Paper Digest

When to use

• Fetch a recent paper digest from arXiv and Hugging Face.
• Produce JSON output for downstream agents.
• Run a local API server when a polling workflow is needed.

Prereqs

• Python 3 and network access.
• LLM access via OPENAI_API_KEY or an OpenAI-compatible provider via LITELLM_API_BASE + LITELLM_API_KEY.
• git is optional for bootstrap; otherwise curl/wget (or Python) is used to download the repo.

Get the code and install

• Preferred: run the bootstrap helper script. It uses git when available or falls back to a zip download.

bash "{baseDir}/scripts/bootstrap.sh"

• Override the clone location by setting PROJECT_DIR.

PROJECT_DIR="$HOME/agentic_paper_digest" bash "{baseDir}/scripts/bootstrap.sh"

Run (CLI preferred)

bash "{baseDir}/scripts/run_cli.sh"

• Pass through CLI flags as needed.

bash "{baseDir}/scripts/run_cli.sh" --window-hours 24 --sources arxiv,hf

Run (API optional)

bash "{baseDir}/scripts/run_api.sh"

• Trigger runs and read results.

curl -X POST http://127.0.0.1:8000/api/run
curl http://127.0.0.1:8000/api/status
curl http://127.0.0.1:8000/api/papers

• Stop the API server if needed.

bash "{baseDir}/scripts/stop_api.sh"

Outputs

• CLI --json prints run_id, seen, kept, window_start, and window_end.
• Data store: data/papers.sqlite3 (under PROJECT_DIR).
• API: POST /api/run, GET /api/status, GET /api/papers, GET/POST /api/topics, GET/POST /api/settings.

Configuration

Config files live in PROJECT_DIR/config. Environment variables can be set in the shell or via a .env file. The wrappers here auto-load .env from PROJECT_DIR (override with ENV_FILE=/path/to/.env).

Environment (.env or exported vars)

• OPENAI_API_KEY: required for OpenAI models (litellm reads this).
• LITELLM_API_BASE, LITELLM_API_KEY: use an OpenAI-compatible proxy/provider.
• LITELLM_MODEL_RELEVANCE, LITELLM_MODEL_SUMMARY: models for relevance and summarization (summary defaults to relevance model if unset).
• LITELLM_TEMPERATURE_RELEVANCE, LITELLM_TEMPERATURE_SUMMARY: lower for more deterministic output.
• LITELLM_MAX_RETRIES: retry count for LLM calls.
• LITELLM_DROP_PARAMS=1: drop unsupported params to avoid provider errors.
• WINDOW_HOURS, APP_TZ: recency window and timezone.
• ARXIV_CATEGORIES: comma-separated categories (default includes cs.CL,cs.AI,cs.LG,stat.ML,cs.CR).
• ARXIV_API_BASE, HF_API_BASE: override source endpoints if needed.
• ARXIV_MAX_RESULTS, ARXIV_PAGE_SIZE: arXiv paging limits.
• MAX_CANDIDATES_PER_SOURCE: cap candidates per source before LLM filtering.
• FETCH_TIMEOUT_S, REQUEST_TIMEOUT_S: source fetch and per-request timeouts.
• ENABLE_PDF_TEXT=1: include first-page PDF text in summaries; requires PyMuPDF (pip install pymupdf).
• DATA_DIR: location for papers.sqlite3.
• CORS_ORIGINS: comma-separated origins allowed by the API server (UI use).
• Path overrides: TOPICS_PATH, SETTINGS_PATH, AFFILIATION_BOOSTS_PATH.