Architecture Walkthrough

Why Unlimited Concurrency Is a Problem

Promise.all([task1(), task2(), ..., task100()]) starts all 100 async operations simultaneously. For network requests, this means 100 concurrent HTTP connections, which exhausts the browser's per-domain connection limit (typically 6 connections in Chrome) and often triggers 429 rate-limit responses from APIs. For CPU-intensive async tasks, it can cause memory spikes from many in-flight responses buffered simultaneously.

Batching into fixed groups (run tasks 1-5, wait for all, then run 6-10) solves the rate problem but is inefficient: if tasks 1-4 finish in 100ms and task 5 takes 2 seconds, the next group cannot start until task 5 completes. The entire group waits for the slowest member.

Sliding Window with Promise.race

The sliding window approach maintains exactly N in-flight promises at all times. When any one finishes, the next task starts immediately, keeping the concurrency at the limit without waiting for an entire batch. Promise.race is the key primitive: it resolves as soon as any one of the provided promises settles, signaling that a slot is free.

The implementation maintains a Set of in-flight promises. Each promise removes itself from the Set when it settles. When the Set reaches the concurrency limit, the loop awaits Promise.race(executing), which yields until one slot opens, then continues to the next task.

Result Collection and Error Handling

Each promise is pushed to a results array immediately when it starts, preserving the original task order. Using Promise.allSettled(results) at the end collects all outcomes (fulfilled or rejected) without short-circuiting on failure. If individual failures should abort the pool, use Promise.all(results) instead, which rejects on the first rejection.

Key Code Explained

async function asyncPool(limit, tasks) {
  const results = [];
  const executing = new Set();

  for (const task of tasks) {
    // Wrap the task so the promise removes itself from executing when done
    const p = Promise.resolve().then(() => task());
    results.push(p);

    const cleanup = p.finally(() => executing.delete(cleanup));
    executing.add(cleanup);

    // If at the limit, wait for any one to finish before continuing
    if (executing.size >= limit) {
      await Promise.race(executing);
    }
  }

  // Wait for all remaining in-flight tasks
  return Promise.allSettled(results);
}

// Usage: fetch 50 users with max 5 simultaneous requests
const userIds = Array.from({ length: 50 }, (_, i) => i + 1);

const tasks = userIds.map((id) => () => fetch(`/api/users/${id}`).then((r) => r.json()));

const results = await asyncPool(5, tasks);

results.forEach((result, i) => {
  if (result.status === 'fulfilled') {
    console.log(`User ${userIds[i]}:`, result.value);
  } else {
    console.warn(`User ${userIds[i]} failed:`, result.reason);
  }
});

// Alternative: simple batch approach (less efficient — waits for slowest in each batch)
async function batchedRequests(ids, batchSize) {
  const results = [];
  for (let i = 0; i < ids.length; i += batchSize) {
    const batch = ids.slice(i, i + batchSize);
    const batchResults = await Promise.all(batch.map((id) => fetchUser(id)));
    results.push(...batchResults);
  }
  return results;
}

The cleanup = p.finally(() => executing.delete(cleanup)) pattern is the critical implementation detail. When a promise settles (fulfilled or rejected), finally runs and removes the wrapper from the executing Set. Promise.race(executing) then resolves because one of the promises in the Set just settled, allowing the for loop to advance to the next task.

Tradeoffs

What Interviewers Actually Check

• Whether you understand why Promise.all with many tasks causes rate limit and connection problems
• Whether you know the difference between batching and sliding window concurrency
• Whether you can explain how Promise.race is used to detect when a slot opens
• Whether you know how to preserve result order while tasks complete out of order
• Whether you can handle individual failures inside the pool without aborting everything

Follow-Up Questions

1. How would you add a timeout to each individual task inside the pool so slow tasks do not block a slot forever?
2. How would you implement backpressure: if tasks are being added faster than they complete, how do you pause the producer?
3. What is the difference between p-limit, p-queue, and a custom asyncPool in terms of features?
4. If one task inside the pool throws synchronously (not an async rejection), how does the current implementation handle it?
5. How would you track per-task progress and report overall completion percentage to the UI?

Common Candidate Mistakes

• Using Promise.all with all tasks at once for a large dataset and not anticipating rate limits or connection exhaustion
• Implementing batch groups that wait for the entire batch before starting the next, not realizing this is less efficient than a sliding window
• Not understanding that Promise.race resolves as soon as any one of the provided promises settles, not all of them
• Losing result order because the pool processes tasks out of order and results are collected in completion order
• Not knowing that Promise.allSettled is needed instead of Promise.all to collect results when individual failures are acceptable

Interview Readiness Checklist

Before you leave this question, make sure you can answer:

• Can you explain why Promise.all with 100 tasks is problematic for network-heavy scenarios?
• Can you write or trace through the sliding window asyncPool implementation?
• Can you explain how the executing Set tracks in-flight promises and when entries are removed?
• Can you describe the difference between batching (groups of N) and sliding window (always N in flight)?
• Can you handle individual task rejections without aborting the entire pool?

Summary

When you have many async tasks and a concurrency limit to respect (API rate limits, browser connection limits, memory constraints), Promise.all with all tasks is too aggressive and simple batching is too conservative. The sliding window pool strikes the right balance: it maintains exactly N in-flight tasks at all times and starts the next task the moment any current task finishes.

The implementation uses a Set to track in-flight promises and Promise.race to detect when one finishes. Each promise wraps itself with .finally() to remove itself from the Set on completion. When the Set reaches the limit, the loop awaits Promise.race(executing), yielding control until a slot opens.

Result collection is decoupled from the concurrency control. Results are pushed to an array in the original task order as each task starts. Promise.allSettled at the end collects all outcomes regardless of individual successes or failures, returning a results array in the same order as the input task list.