azure-search-openai-demo: Instructions file for Codex

Instructions for Coding Agents

This file contains instructions for developers working on the Azure Search and OpenAI demo application. It covers the overall code layout, how to add new data, how to add new azd environment variables, how to add new developer settings, and how to add tests for new features.

Always keep this file up to date with any changes to the codebase or development process. If necessary, edit this file to ensure it accurately reflects the current state of the project.

Overall code layout

• app: Contains the main application code, including frontend and backend.
  • app/backend: Contains the Python backend code, written with Quart framework.
    • app/backend/approaches: Contains the different approaches
      • app/backend/approaches/approach.py: Base class for all approaches
      • app/backend/approaches/chatreadretrieveread.py: Chat approach, includes query rewriting step first
      • app/backend/approaches/promptmanager.py: Manages loading and rendering of Jinja2 prompt templates
      • app/backend/approaches/prompts/query_rewrite.system.jinja2: Jinja2 template used to rewrite the query based off search history into a better search query
      • app/backend/approaches/prompts/chat_query_rewrite_tools.json: Tools used by the query rewriting prompt
      • app/backend/approaches/prompts/chat_answer.system.jinja2: Jinja2 template for the system message used by the Chat approach to answer questions
      • app/backend/approaches/prompts/chat_answer.user.jinja2: Jinja2 template for the user message used by the Chat approach, including sources
    • app/backend/prepdocslib: Contains the document ingestion library used by both local and cloud ingestion
      • app/backend/prepdocslib/blobmanager.py: Manages uploads to Azure Blob Storage
      • app/backend/prepdocslib/cloudingestionstrategy.py: Builds the Azure AI Search indexer and skillset for the cloud ingestion pipeline
      • app/backend/prepdocslib/csvparser.py: Parses CSV files
      • app/backend/prepdocslib/embeddings.py: Generates embeddings for text and images using Azure OpenAI
      • app/backend/prepdocslib/figureprocessor.py: Generates figure descriptions for both local ingestion and the cloud figure-processor skill
      • app/backend/prepdocslib/fileprocessor.py: Orchestrates parsing and chunking of individual files
      • app/backend/prepdocslib/filestrategy.py: Strategy for uploading and indexing files (local ingestion)
      • app/backend/prepdocslib/htmlparser.py: Parses HTML files
      • app/backend/prepdocslib/integratedvectorizerstrategy.py: Strategy using Azure AI Search integrated vectorization
      • app/backend/prepdocslib/jsonparser.py: Parses JSON files
      • app/backend/prepdocslib/listfilestrategy.py: Lists files from local filesystem or Azure Data Lake
      • app/backend/prepdocslib/mediadescriber.py: Interfaces for describing images (Azure OpenAI GPT-4o, Content Understanding)
      • app/backend/prepdocslib/page.py: Data classes for pages, images, and chunks
      • app/backend/prepdocslib/parser.py: Base parser interface
      • app/backend/prepdocslib/pdfparser.py: Parses PDFs using Azure Document Intelligence or local parser
      • app/backend/prepdocslib/searchmanager.py: Manages Azure AI Search index creation and updates
      • app/backend/prepdocslib/servicesetup.py: Shared service setup helpers for OpenAI, embeddings, blob storage, etc.
      • app/backend/prepdocslib/strategy.py: Base strategy interface for document ingestion
      • app/backend/prepdocslib/textparser.py: Parses plain text and markdown files
      • app/backend/prepdocslib/textprocessor.py: Processes text chunks for cloud ingestion (merges figures, generates embeddings)
      • app/backend/prepdocslib/textsplitter.py: Splits text into chunks using different strategies
    • app/backend/app.py: The main entry point for the backend application.
  • app/functions: Azure Functions used for cloud ingestion custom skills (document extraction, figure processing, text processing). Each function bundles a synchronized copy of prepdocslib; run python scripts/copy_prepdocslib.py to refresh the local copies if you modify the library.
  • app/frontend: Contains the React frontend code, built with TypeScript, built with vite.
    • app/frontend/src/api: Contains the API client code for communicating with the backend.
    • app/frontend/src/components: Contains the React components for the frontend.
    • app/frontend/src/locales: Contains the translation files for internationalization.
      • app/frontend/src/locales/da/translation.json: Danish translations
      • app/frontend/src/locales/en/translation.json: English translations
      • app/frontend/src/locales/es/translation.json: Spanish translations
      • app/frontend/src/locales/fr/translation.json: French translations
      • app/frontend/src/locales/it/translation.json: Italian translations
      • app/frontend/src/locales/ja/translation.json: Japanese translations
      • app/frontend/src/locales/nl/translation.json: Dutch translations
      • app/frontend/src/locales/ptBR/translation.json: Portuguese translations
      • app/frontend/src/locales/tr/translation.json: Turkish translations
    • app/frontend/src/pages: Contains the main pages of the application
• infra: Contains the Bicep templates for provisioning Azure resources.
• evals: Contains evaluation configs, datasets, and results.
  • evals/results: Contains raw per-run eval output folders. Use descriptive setup-based names for repeated runs, such as gpt54-low-top5-run1.
  • evals/results_summaries: Contains derived grouped summaries such as baseline.json and baseline.md.
  • evals/results_comparisons: Reserved for derived candidate-vs-baseline comparison artifacts.
  • evals/eval_compare.py: Compares eval result folders and reports averages, confidence intervals, and paired significance tests.
• tests: Contains the test code, including e2e tests, app integration tests, and unit tests.