vector-search-patterns

Vector Search Patterns — Akashic Context Sprint 1

Design Decision (D1)

No sqlite-vec extension. Cosine similarity implemented in TypeScript directly.

Why: Embeddings already stored as JSON in chunks.embedding column. sqlite-vec has platform loading issues. In-process is sufficient for ~2000 chunks/user.

cosine Similarity Function

Add as module-level function in storage.ts (NOT exported — internal utility):

function cosineSimilarity(a: number[], b: number[]): number {
  let dot = 0, magA = 0, magB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    magA += a[i] * a[i];
    magB += b[i] * b[i];
  }
  const mag = Math.sqrt(magA) * Math.sqrt(magB);
  if (mag === 0) return 0;
  return dot / mag;
}

Mathematical properties (use in tests):

• cosineSimilarity([1,0], [1,0]) → 1.0 (identical)
• cosineSimilarity([1,0], [0,1]) → 0.0 (orthogonal)
• cosineSimilarity([1,0], [-1,0]) → -1.0 (opposite)

searchVectorInProcess() in storage.ts

Add alongside existing searchVector() method:

searchVectorInProcess(params: SearchVectorParams): VectorSearchResult[] {
  let sql = `
    SELECT id, path, source, start_line as startLine, end_line as endLine, text, embedding
    FROM chunks
  `;
  const queryParams: unknown[] = [];

  if (params.source) {
    sql += " WHERE source = ?";
    queryParams.push(params.source);
  }

  const rows = this.db.prepare(sql).all(...queryParams) as Array<{
    id: string; path: string; source: string;
    startLine: number; endLine: number; text: string; embedding: string;
  }>;

  const queryEmb = params.embedding;

  return rows
    .map(row => {
      let chunkEmb: number[];
      try {
        chunkEmb = JSON.parse(row.embedding) as number[];
      } catch {
        return null;
      }
      const similarity = cosineSimilarity(queryEmb, chunkEmb);
      return {
        id: row.id,
        path: row.path,
        source: row.source,
        startLine: row.startLine,
        endLine: row.endLine,
        text: row.text,
        distance: 1 - similarity, // Lower is better (consistent with searchVector interface)
      };
    })
    .filter((r): r is VectorSearchResult => r !== null && r.distance <= (1 - 0.3))
    .sort((a, b) => a.distance - b.distance)
    .slice(0, params.limit);
}