stale-cache-external-service-recovery

Stale Cache Recovery When External Service Loses Data

Problem

When using your database as a cache/idempotency layer (see local-cache-idempotency-fallback), a new failure mode emerges: the external service loses or resets its data, but your cache still has the old values. Operations fail with misleading errors because the cached identifiers don't exist on the external service anymore.

Context / Trigger Conditions

• Token refresh fails with 404 "User with pubkey/id not found"
• Operations fail with "Invalid or expired token" but the real issue is missing user
• External service was reset, migrated, or lost data
• Cached identifiers (pubkeys, API keys, user IDs) are valid in cache but not on service
• Error message is misleading - suggests token issue when it's actually a missing resource

Solution

Detect the specific "not found" case and handle it separately from other errors:

if (cached) {
  pubkey = cached.pubkey;
  token = cached.token;

  try {
    // Try to refresh/validate the cached credentials
    const freshToken = await externalApi.getTokenByPubkey(pubkey);
    token = freshToken;
    console.log(`Pubkey: ${pubkey} (token refreshed)`);
  } catch (error) {
    const errMsg = error instanceof Error ? error.message : String(error);

    // CRITICAL: Detect "not found" specifically - cache is stale
    if (errMsg.includes("not found") || response.status === 404) {
      console.log(`Cached identifier not on external service, recreating...`);

      // 1. Delete stale cache entry
      await db.deleteUser(userId);

      // 2. Create fresh resource on external service
      const result = await externalApi.createUser(userId, username);
      pubkey = result.pubkey;
      token = result.token;
      console.log(`Pubkey: ${pubkey} (recreated)`);

      // 3. Cache new values
      await db.saveUser({ userId, pubkey, token });
    } else {
      // Other error (e.g., user claimed account, rate limit) - can't proceed
      console.log(`Refresh failed: ${errMsg}`);
      continue; // or throw
    }
  }
}