memory-optimization

MoFlo Memory Optimization

When the default moflo memory settings stop being enough — past ~100k entries, or when p95 search latency climbs — these are the levers.

HNSW Parameters

HNSW has three knobs. They trade build time, query time, memory, and recall.

import { HNSWIndex } from 'moflo/dist/src/cli/memory/index.js';

const index = new HNSWIndex({
  dimensions: 1536,        // must match your embedding model
  maxElements: 1_000_000,  // pre-allocated capacity
  M: 16,                   // graph connectivity (default 16)
  efConstruction: 200,     // build-time search width (default 200)
  metric: 'cosine',        // 'cosine' | 'l2' | 'ip'
});

Knob	Higher	Lower	When to change
M	better recall, more RAM (~2×M pointers per point)	less RAM, worse recall	Bump to 32–64 if recall@10 < 0.95; drop to 8 if memory-bound
efConstruction	better index quality, slower build	faster build, worse queries	200–400 is sweet spot; only lower in test fixtures
ef (search-time, passed to search())	better recall, slower queries	faster queries, worse recall	Start at 2×k, raise until recall plateaus

Rule of thumb: M and efConstruction are set once. ef is the runtime dial.

Quantization

moflo memory supports scalar quantization (Float32 → Int8) for a ~4× memory reduction with a ~1-2% recall hit. Turn it on when the index doesn't fit comfortably in RAM.

const index = new HNSWIndex({
  dimensions: 1536,
  maxElements: 5_000_000,
  quantization: {
    enabled: true,
    type: 'scalar',   // scalar (Int8) is the supported path
    rebuildThreshold: 10_000,
  },
});

Measure recall before/after on your own query distribution — public benchmarks don't predict your domain.

Batch Operations

Single-entry writes pay the HNSW insert cost per call. For bulk ingest, batch:

const entries: Array<[string, Float32Array]> = buildCorpus();

// Parallelise at the adapter level; don't await sequentially.
await Promise.all(
  entries.map(([id, vec]) => index.addPoint(id, vec))
);

// Or via MCP for moflo-native batch into .swarm/memory.db:
await mcp.memory_store(/* … */);  // upsert: true + Promise.all is fine