rest-api-optimistic-update-race-condition

REST API Optimistic Update Race Condition

Problem

When an application uses a dual architecture with:

• Write path: Direct database writes or WebSocket/event publishing
• Read path: REST API backed by async indexing (ClickHouse, Elasticsearch, etc.)

Content can "disappear" after publishing because the read path hasn't indexed the new content yet, but the UI re-fetches from the REST API expecting fresh data.

Context / Trigger Conditions

• User publishes content (video, post, comment, etc.)
• Content briefly appears, then disappears on refresh or navigation
• REST API returns N items when N+1 expected
• Logs show successful write but stale read
• Backend uses async indexing (ClickHouse, Elasticsearch, Algolia, etc.)
• Symptoms worsen under load or with slow indexing pipelines

Log pattern example:

✅ Event successfully published to relays
📊 Fetching from REST API: found 59 videos  // Expected 60!

Solution

Option 1: Optimistic State Update (Recommended)

Instead of re-fetching from REST API after publishing, optimistically add the new content directly to the local state:

// BEFORE (broken): Re-fetch from potentially stale API
void onNewContentPublished(Content newContent) {
  refreshFromRestApi(); // API hasn't indexed yet = stale data
}

// AFTER (fixed): Optimistic local update
void onNewContentPublished(Content newContent) {
  final currentItems = state.items;

  // Skip if duplicate
  if (currentItems.any((item) => item.id == newContent.id)) return;

  // Add to front of list (most recent first)
  final updatedItems = [newContent, ...currentItems];

  state = state.copyWith(
    items: updatedItems,
    lastUpdated: DateTime.now(),
  );
}

Option 2: Hybrid Approach

For long-lived sessions, periodically reconcile local state with REST API:

void onNewContentPublished(Content newContent) {
  // Immediate: optimistic update
  _addToLocalState(newContent);

  // Delayed: reconcile with API after indexing lag
  Future.delayed(Duration(seconds: 30), () {
    _reconcileWithRestApi();
  });
}