ndk-batch-event-queries

NDK Batch Event Queries

Problem

When checking if many Nostr events exist (e.g., 300 videos for a user), querying one-by-one causes hundreds of sequential relay round-trips, making the operation extremely slow (minutes instead of seconds).

Context / Trigger Conditions

• Loop with await fetchEvents() inside, checking events individually
• Processing takes minutes when it should take seconds
• Checking existence of addressable events (Kind 30000-39999) by d-tag
• Need to find which items from a list already exist on a relay

Solution

NDK's fetchEvents filter accepts arrays for most fields. Instead of:

// SLOW: 300 sequential queries
for (const id of vineIds) {
  const exists = await ndk.fetchEvents({
    kinds: [34236],
    authors: [pubkey],
    "#d": [id],  // Single value
  });
}

Use a batch query:

// FAST: 1-3 queries (chunked if needed)
const CHUNK_SIZE = 100;  // Relays may limit query size
const existingIds = new Set<string>();

for (let i = 0; i < vineIds.length; i += CHUNK_SIZE) {
  const chunk = vineIds.slice(i, i + CHUNK_SIZE);
  const events = await ndk.fetchEvents({
    kinds: [34236],
    authors: pubkeys,  // Can also be an array
    "#d": chunk,       // Array of d-tag values
  });

  for (const event of events) {
    const dTag = event.tags.find(t => t[0] === "d");
    if (dTag?.[1]) existingIds.add(dTag[1]);
  }
}

// O(1) lookup in processing loop
for (const id of vineIds) {
  if (existingIds.has(id)) continue;  // Skip existing
  // Process new items...
}

Key Points

1. Array filters: #d, #p, #e, authors all accept arrays
2. Chunk size: Use 50-100 items per query to avoid relay limits
3. Multiple authors: Pass array of pubkeys if checking across users
4. Extract results: Parse the d-tag from returned events to build a Set

Verification

• Processing time drops from minutes to seconds
• Total relay connections decrease dramatically
• Same results as individual queries (verified by comparison)