HotelAI: Instructions file for Codex

AGENTS.md

System overview

• src/ingestion/: PDF loading/segmentation and structured Groq/Gemini-reviewed extraction.
  • pdf_parser.py: pdfplumber-based hotel-block segmentation, OCR cleanup, and word bounding boxes (used as the reference .words source for the visual-ratings fallback).
  • pymupdf_parser.py: font-size-aware header extractor; canonical nome/localita source for structured_extractor. The two flat exclusion word-lists (_EXCLUDED_HEADER_TEXT, _EXCLUDED_BADGE_WORDS) live as one-word-per-line .txt files in pdf_artifacts/, not as Python constants — edit those files to add/remove excluded words, no code change needed. _STOP_MARKERS (regex) and _LIGATURE_GLYPHS (character-substitution rule) stayed in code: they're parsing logic, not data.
  • structured_extractor.py: canonical HotelRecord extraction and CSV/JSONL export.
• src/search/: embeddings, ChromaDB indexing, hybrid retrieval and Markdown formatting.
  • vector_store.py: GeminiEmbedder, offline embedder and persistent Chroma index. build_index_from_csv reads hotels_data.csv back via structured_extractor.read_csv and is the real Part-1-to-Part-2 bridge — build_index itself takes only records, no PDF blocks, since the document text embedded per hotel is record.source.raw_text, already present in the CSV's source column.
  • retriever.py: maximum-five-result search and ranking.
• src/rag/rag_engine.py: RAGEngine/answer_query conversational synthesis over HotelRetriever results; tries Groq first, falls back to Gemini, then to the fixed fallback string FALLBACK_MESSAGE. Not yet wired into mcp_server.
• src/mcp_server/server.py: canonical FastMCP server and search_hotels(query) tool. Does not yet expose the RAG engine's natural-language synthesis, only raw retrieval.
• src/api/main.py: FastAPI HTTP layer for the React frontend — POST /api/chat (RAGEngine.answer_query), GET /api/hotels (reads settings.hotel_records_path, i.e. data/processed/hotels_data.jsonl, the canonical HotelRecord-schema export — not the legacy hotels.jsonl), GET /api/health (Chroma connectivity/count + Groq/Gemini key presence). Retriever/RAG-engine access is via Depends(get_retriever)/Depends(get_rag_engine), overridable in tests; the real Chroma index is only built in the lifespan startup hook, so tests never touch it unless they opt in. CORS is open to http://localhost:5173 (Vite dev server) only. Run: uvicorn api.main:app --reload --port 8000 (PYTHONPATH=src, from repo root).
• frontend/: Vite + React chatbot UI (dark theme, ChatGPT/Claude-style layout). Talks to the backend via /api/*, proxied to localhost:8000 by vite.config.js in dev — CORS is also configured on the backend as a second line of defense. src/components/: Sidebar.jsx (status, 5 quick queries, hotel catalog accordion), ChatArea.jsx (hero view, message bubbles, page-range citation badges, typing indicator), InputBar.jsx. src/styles/app.css holds the whole design (CSS variables for the zinc/emerald palette). No markdown renderer is wired in — assistant answers render as white-space: pre-wrap plain text, so literal **bold**/table-pipe markup from the LLM is visible as-is; add react-markdown if that needs to render properly. Run: cd frontend && npm install && npm run dev (port 5173). .claude/launch.json has a frontend config for the preview_start tool.
• src/hotelai/: canonical settings/Settings (config.py), HotelRecord schema (schemas.py), centralized logging setup (logging_setup.py), prompts/ (LLM prompt text as .md files plus load_prompt(name), see line below), and server.py (deprecated compat wrapper, see line below). Renamed from fde_hotel_rag; the project folder itself is still named fde_hotel_rag pending a manual rename by the user (see Known gotchas). Formerly also held extractors//storage/ subpackages — deleted, they contained no source, only stale __pycache__.
• src/hotelai/prompts/: all LLM prompt text lives here as .md files, not inline in Python — load_prompt(name) reads <name>.md and strips it; callers .format(...) it themselves when the template has placeholders (conversational_rag.md takes {fallback_message}, structured_review.md takes {example}, the dynamically-built HotelReview JSON sample). visual_ratings.md has no placeholders. Editing a prompt's wording only ever means editing the .md file.
• scripts/run_pipeline.py: primary ingestion entry point; --offline skips API calls and indexing.
• tests/: unit/integration-style tests for parsing, extraction, Chroma, retrieval, MCP and the FastAPI layer (test_api.py, 100% mocked via app.dependency_overrides).
• scripts/compare_pdf_extractors.py: comparison-only PyMuPDF extraction benchmark.