Architecture Walkthrough

Retryable vs Non-Retryable Errors

The first and most important decision in retry logic is which errors should trigger a retry. 5xx errors (500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout) are transient: they indicate a server-side problem that may resolve itself. Network errors (failed to fetch, ECONNREFUSED, ETIMEDOUT) are also transient.

4xx errors are not transient. A 400 Bad Request means the request payload is malformed. A 401 means the credentials are wrong or expired. A 403 means the resource is forbidden. A 404 means the endpoint does not exist. Retrying any of these will produce the same error every time and only delays the error propagation to the caller.

Exponential Backoff

Fixed-delay retry (wait 1 second between every attempt) is naive. If a server is overwhelmed and 500 clients all back off for 1 second and retry simultaneously, the server faces the same burst 1 second later. Exponential backoff increases the delay geometrically: 100ms, 200ms, 400ms, 800ms. This reduces the load on the server on each successive retry cycle.

The standard formula is delay = baseDelay * 2^attemptNumber. A maximum delay cap prevents the backoff from growing to impractical durations on high retry counts. A typical cap is 30 seconds or the total SLA budget for the operation.

Jitter

Even with exponential backoff, if all clients started at the same time they will still retry in synchronized bursts at 100ms, 200ms, 400ms. Jitter adds randomness to the delay to spread retries across time. Full jitter replaces the deterministic delay with a random value between 0 and the calculated backoff: Math.random() * delay. Decorrelated jitter (Math.min(cap, random(base, prev * 3))) produces better distribution but the implementation is more complex. AWS recommends full jitter for most use cases.

Key Code Explained

function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

async function fetchWithRetry(url, options = {}, retries = 3, baseDelay = 100) {
  let lastError;

  for (let attempt = 0; attempt <= retries; attempt++) {
    try {
      const res = await fetch(url, options);

      // Do not retry 4xx errors — they are not transient
      if (res.status >= 400 && res.status < 500) {
        throw Object.assign(new Error(`Client error: ${res.status}`), {
          status: res.status,
          retryable: false,
        });
      }

      if (!res.ok) {
        // 5xx — transient, throw to trigger retry
        throw Object.assign(new Error(`Server error: ${res.status}`), {
          status: res.status,
          retryable: true,
        });
      }

      return await res.json();
    } catch (err) {
      // Non-retryable: propagate immediately
      if (err.retryable === false) throw err;

      lastError = err;

      if (attempt < retries) {
        // Exponential backoff with full jitter and 30s cap
        const exponential = baseDelay * 2 ** attempt;
        const capped = Math.min(exponential, 30_000);
        const jittered = Math.random() * capped;
        console.warn(`Attempt ${attempt + 1} failed. Retrying in ${Math.round(jittered)}ms...`);
        await sleep(jittered);
      }
    }
  }

  throw lastError; // all retries exhausted — propagate the last error
}

// Usage
try {
  const data = await fetchWithRetry('/api/jobs', { method: 'GET' }, 4, 100);
  console.log(data);
} catch (err) {
  console.error('All retries exhausted:', err.message);
}

// Adding per-attempt timeout with AbortController
async function fetchWithTimeout(url, options = {}, timeoutMs = 5000) {
  const controller = new AbortController();
  const id = setTimeout(() => controller.abort(), timeoutMs);
  try {
    return await fetch(url, { ...options, signal: controller.signal });
  } finally {
    clearTimeout(id);
  }
}

// Combine: retry with per-attempt timeout
async function robustFetch(url, retries = 3, timeoutMs = 5000) {
  return fetchWithRetry(
    url,
    { signal: AbortSignal.timeout(timeoutMs) }, // per-attempt timeout
    retries,
    100,
  );
}

The err.retryable = false flag is the key design choice. By tagging non-retryable errors at the point of detection (when the status code is 4xx), the catch block can propagate immediately without needing to re-examine the status code or parse the error message. This makes the retry logic independent of HTTP specifics.

Tradeoffs

What Interviewers Actually Check

• Whether you know to skip retries for 4xx errors
• Whether you can implement the exponential backoff formula correctly
• Whether you know what jitter is and why it prevents thundering herd
• Whether you cap the maximum delay
• Whether you propagate the last error after all retries are exhausted rather than swallowing it

Follow-Up Questions

1. How would you implement a circuit breaker on top of the retry logic to stop retrying after a threshold of consecutive failures?
2. How would you make the retry logic work with the p-retry library and what does it add over a manual implementation?
3. How would you log each retry attempt to an observability system (Datadog, Sentry) for debugging production failures?
4. If you need to retry a mutation (POST, PUT) and the server is not idempotent, what risk does retry introduce?
5. How does AbortSignal.timeout(ms) differ from creating your own AbortController and calling abort() in a setTimeout?

Common Candidate Mistakes

• Retrying 401 or 403 errors, which will fail every time and only delay the caller receiving the error
• Using a fixed delay and not knowing it causes synchronized retry bursts when many clients fail simultaneously
• Not adding jitter and not being able to explain what thundering herd is or why it matters
• Not capping the maximum delay, allowing a 10-retry configuration to wait over 800 seconds total
• Swallowing the error after all retries and returning undefined or null instead of throwing the last error

Interview Readiness Checklist

Before you leave this question, make sure you can answer:

• Can you implement fetchWithRetry with exponential delay and 4xx bypass from memory?
• Can you explain why jitter is added and what thundering herd means in plain terms?
• Can you explain the difference between a per-attempt timeout and a total timeout across all retries?
• Can you add a maximum delay cap to prevent unbounded backoff?
• Can you modify the implementation to collect all attempt errors rather than only the last?

Summary

Retry logic with exponential backoff is a standard pattern for handling transient failures in network-dependent systems. The core idea is: retry only errors that are likely to resolve by themselves (5xx, network errors), increase the wait between retries geometrically so the server has time to recover, and add jitter to spread retries across time when many clients experience the same failure.

The implementation iterates up to retries + 1 times. On each failed attempt, the delay is calculated as baseDelay * 2^attempt, capped at a maximum, and then multiplied by a random factor between 0 and 1 (full jitter). Non-retryable errors (4xx) are tagged and propagated immediately without entering the retry loop. After all retries are exhausted, the last captured error is thrown to the caller.

Combining retry with a per-attempt timeout (using AbortController) prevents a single slow connection from blocking the entire retry window. The total maximum time for an operation is bounded by (retries + 1) * timeoutMs + totalBackoffDelay, which should fit within the SLA budget of the calling operation.