parkour-mcp CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project

parkour-mcp — an MCP server providing a content exploration and research synthesis pipeline. Uses clean first-party APIs to surface and explore web content without summarization. Integrates Kagi, Semantic Scholar, arXiv, deps.dev, IETF, GitHub, MediaWiki, Reddit, Discourse, YouTube, and DOI resolution APIs into a unified tool suite for Claude Code and Claude Desktop.

Commands

# Run mocked unit tests (default, excludes live tests)
uv run pytest

# Run a single test file or specific test
uv run pytest tests/test_arxiv.py
uv run pytest tests/test_arxiv.py::test_function_name

# Run live integration tests (hits real endpoints)
uv run pytest -m live

# Regenerate README examples (live endpoints + Reddit fixtures)
uv run python3 scripts/regenerate_readme_examples.py

# Pack Claude Desktop Extension bundle
just pack

# Preview next release (version + CHANGELOG entry), no writes
just release-preview

Architecture

Module Layout (parkour_mcp/)

• __init__.py — MCP server entry point. Registers 13 always-on tools, plus 1 optional (SemanticScholar, gated by S2_ACCEPT_TOS), with profile-specific names (PascalCase for code, snake_case for desktop). Description templates have placeholders replaced at registration time.

• detection.py — Pure, stdlib-only URL/identifier classification: the _detect_*_url predicates, regexes, ID extractors, and RedditPageType for arXiv, DOI, Semantic Scholar, IETF, and Reddit. Imported at module top by fetch_direct.py, _pipeline.py, and kagi.py (and by the source modules for their own internal use). Kept dependency-free so the fast-path dispatchers and sibling tools can classify a URL without loading a source module's transport stack (httpx, wreq). Also hosts HFUrlMatch / _detect_hf_url for HuggingFace: gated, private, and nonexistent are all response properties on the Hub (it returns an identical 401 for the three), so nothing about classifying a Hub URL can consult a token and detection stays pure. GitHub is the exception that proves the rule — its detection stays in github.py because _detect_github_url consults _get_github_token() to auth-gate discussion URLs, so it is not stateless. MediaWiki and Discourse detect at fetch time, not from the URL.
• _pipeline.py — Shared processing layer. Owns the fast-path dispatch chain, multi-entry caching (_WikiCache LRU, _PageCache 2Q), slicing, BM25 search, and section filtering. URL detection itself lives in detection.py. Also owns unspaced-script indexing, the character n-gram path that makes BM25 work for Japanese, Chinese, Korean, Thai and their neighbours: tantivy's default analyzer breaks on non-alphanumerics only, so a whole Japanese clause indexes as one term and every query over such a page returned nothing. _unspaced_runs splits the text into runs of those scripts and each indexed field gains an _unspaced shadow carrying the same content under an n-gram analyzer, which is Lucene's dictionary-free CJK approach — a segmenter (lindera, jieba) is not reachable from tantivy-py without a from-source build, and NTCIR evaluations find character bigrams competitive with one regardless. youtube.py imports the same three helpers for its transcript index, so any index over a schema declaring an _unspaced field must be constructed via _new_search_index, which registers the analyzer; a bare tantivy.Index(schema) accepts no documents.
• markdown.py — HTML→markdown conversion. Two converters, deliberately: the generic path (html_to_markdown) runs the Rust-backed htmd, ~12–47× faster than the markdownify implementation it replaced in da4f54a while recovering +27.6% content on table-heavy pages; the custom TextOnlyConverter (markdownify + BS4) is retained solely for mediawiki.py, which applies BS4 transforms (navbox pruning, math extraction, footnote rewriting) before converting and so cannot hand raw HTML to htmd. _wrap_bare_pre_in_code runs before conversion because htmd fences only <pre><code>: Sphinx and Pygments emit <pre><span class="k">… with no <code> wrapper, so without it every # comment in a Python example lands at column zero and reads as a heading (31 phantom level-1 sections on Django's queryset reference, 21 on PEP 8). Two further pre-conversion passes repair headings that htmd would otherwise lose to the markup around them, each the other's mirror image: _unwrap_heading_anchors drops an <a> wrapped around a heading, which straddles a block boundary and leaves orphan link syntax on the heading line (site mastheads and card links); _flatten_heading_blocks replaces a block element nested inside a heading with a space, since it ends the # line and pushes the title down into body prose (Mintlify's docs theme, which put all 30 headings of Claude Code's own memory page out of reach). Both tag sets are measured against htmd rather than taken from the HTML spec — the two disagree in both directions. Section extraction with fuzzy slug matching, preceded by _promote_list_headings: accordion and disclosure widgets nest each section's <h2> in an <li>, so the heading is lifted to line start and the item body dedented, or the line-start anchor misses every one of them. An explicit list marker is required, which is what keeps # comment lines in unfenced indented code blocks from being read as headings. Content fencing. Semantic truncation for markdown, hard truncation for structured formats. Also hosts _plaintext_presplit, the generic line-oriented cache presplitter with the issue-#6 circuit breaker: it has no source-specific behavior and more than one tool needs it (github.py#_blob_presplit uses it as the fallback under tree-sitter, huggingface.py uses it directly since none of the formats that tool reads have a grammar registered), so it lives at the common ancestor rather than in either tool.
• shelf.py — Research shelf implementation. All public methods guarded by asyncio.Lock.