marklogic-rag

RAG on MarkLogic 12

Requires MarkLogic 12+. Check for an existing vector view first with ml_views_list and ml_schema_get_tde.

Pick a paradigm

Situation	Approach
No embeddings, fast prototype	Lexical RAG
Embeddings available, simple queries	Pure vector (cosine)
Mixed vocabulary + semantic queries	ANN hybrid (ANN + BM25)
Taxonomy / knowledge graph available	Graph + Lexical
Highest precision, full infrastructure	Graph + Vector

The graph layer composes with any paradigm — it only supplies a URI scope filter, so apply it before the search or ANN step.

1. Storing embeddings

Store the embedding as a JSON array of floats in an embedding field, sibling to the content fields. Carry embeddingModel, embeddingDim, and — when chunking — chunkIndex, chunkText, sourceUri.

Chunking strategy by domain: legal contracts → by clause; news articles → whole-document or paragraph; product manuals → by section.

⚠ Generate embeddings in your application tier, not in MarkLogic. xdmp.httpPost to external HTTPS embedding APIs commonly fails with tlsv1 unrecognized name (an SSL SNI limitation in MarkLogic's Java SSL client). Produce the float array outside, then store it with flux_import or ml_document_put.

2. The TDE vector column

{
  "template": {
    "context": "/",
    "collections": ["my-collection"],
    "rows": [{
      "schemaName": "my_schema",
      "viewName": "my_view",
      "columns": [
        { "name": "uri",       "scalarType": "string", "val": "xdmp:node-uri(.)" },
        { "name": "content",   "scalarType": "string", "val": "chunkText" },
        { "name": "embedding", "scalarType": "vector", "val": "array-node('embedding')",
          "dimension": "1536", "invalidValues": "reject" }
      ]
    }]
  }
}

Four rules that each cause a distinct failure:

• "scalarType": "vector" — not "vec:vector", not "scalar". Either of those gives TDE-INVALIDTEMPLATENODEVAL.
• "val": "array-node('embedding')" — plain "val": "embedding" atomises the array into N individual number nodes and throws XDMP-CAST.
• "dimension" is required and must match the model output exactly, or queries fail with XDMP-DIMMISMATCH.
• "invalidValues": "reject" — skips malformed documents instead of failing the whole view.