dvcon_ai_library AGENTS.md

AGENTS.md

This repository contains a full-stack DVCon paper search and chat application. This file is the handoff guide for future AI coding agents.

Purpose

Build and maintain a web app that:

• downloads DVCon papers from https://dvcon-proceedings.org/
• stores raw PDFs under data/paper/
• extracts markdown, images, and metadata under data/
• supports keyword search and semantic search
• supports grounded chat over selected papers
• provides a professional React frontend and a FastAPI backend

User Preferences

• Use uv for Python dependency management and Python commands.
• Use snake_case and 4-space indentation.
• Do not commit secrets or the full generated runtime corpus. A curated example corpus is acceptable only when explicitly requested by the user.
• Default interpretation of a change request: do the implementation work, update the code, update AGENTS.md, and update PROGRESS.md unless the user explicitly narrows the scope.

Current Architecture

• Backend: FastAPI, SQLModel, SQLite FTS5, ChromaDB
• Frontend: React, TypeScript, Vite, MUI
• PDF extraction: PyMuPDF, pymupdf4llm
• Metadata enrichment: local GROBID sidecar producing TEI XML, enabled by default
• Scraping: httpx, BeautifulSoup4
• Chat: OpenAI Responses API via configurable OPENAI_BASE_URL and OPENAI_API_KEY
• Embeddings: local sentence-transformers model via torch
• Local embedding device: CUDA preferred, CPU fallback
• Agent access: MCP server (mcp SDK, stdio transport) reusing the service layer; Claude plugin marketplace + agent skill mirror the same tool surface

Key Product Requirements

• Only index DVCon items whose detail page says Type: Paper and Format: pdf.
• Save PDFs at data/paper/{year}/{location}/{slug}.pdf.
• Save markdown at data/markdown/{year}/{location}/{slug}.md.
• Save extracted images at data/markdown/{year}/{location}/images/{slug}/.
• Save raw GROBID TEI at data/tei/{year}/{location}/{slug}.tei.xml when available.
• Extract and persist metadata such as:
  • title
  • authors
  • affiliations / company names
  • abstract
  • references
  • year
  • conference location
• Left panel tabs:
  • Search Results
  • PDF
  • Markdown
  • Metadata Graph
• Right panel:
  • chat transcript
  • input box
  • Enter submits
  • Shift+Enter inserts newline
• Metadata Graph nodes are clickable (per node type):
  • author / company → jump to Search Results filtered by that name (keyword FTS match against authors / affiliations)
  • conference → jump to Search Results filtered by that conference's year + location (precise, not free-text)
  • reference → if the citation's normalized title resolves to an in-corpus paper, jump to that paper's PDF tab; unresolved references render non-clickable
  • paper (the active paper) → no-op (already viewing it)
  • The graph tab stays mounted across author/company/conference clicks (graph_query is keyed on active_paper_id, which doesn't change), so the user returns to the graph by re-clicking the Metadata Graph tab.