Beyond Promise.all: Advanced TypeScript Async Await Patterns for High-Throughput Systems

There are two async patterns that kill production systems, and most codebases have at least one of them.

The first is the sequential for...of await — safe, readable, and catastrophically slow when you have hundreds of independent tasks. The second is Promise.all(items.map(...)) — fast, but completely unbounded. Point it at 10,000 items and you'll exhaust your socket pool, trigger rate limits on downstream services, and potentially crash the Node process with an OOM error.

The fix is a concurrency-controlled worker pool: a fixed number of async workers that pull from a shared queue, so you're always processing N tasks in parallel and never more.

Why Unbounded Concurrency Fails

When you fire Promise.all on a large array, you instantiate all N promises simultaneously. Node.js has a finite socket pool managed by libuv. Blow past it and requests start throwing EMFILE. Hit a downstream API hard enough and you get HTTP 429s. Allocate thousands of large promise objects at once and V8's garbage collector can fall behind, leading to heap pressure that degrades the entire process.

The problem isn't Promise.all itself — it's using it on arrays of unknown or dynamic length without any throttle.

The Concurrent Worker Pool

The pattern is straightforward: spin up exactly N workers, each of which atomically claims the next index from a shared counter and processes it, then loops until the queue is empty.

export type SettledResult<R> =
  | { status: 'fulfilled'; value: R }
  | { status: 'rejected'; reason: Error };
 
export async function concurrentMapSettled<T, R>(
  items: T[],
  concurrencyLimit: number,
  fn: (item: T, index: number) => Promise<R>
): Promise<SettledResult<R>[]> {
  const results: SettledResult<R>[] = new Array(items.length);
  let nextIndex = 0;
 
  async function worker(): Promise<void> {
    while (nextIndex < items.length) {
      const i = nextIndex++;
      try {
        results[i] = { status: 'fulfilled', value: await fn(items[i], i) };
      } catch (error) {
        results[i] = { status: 'rejected', reason: error instanceof Error ? error : new Error(String(error)) };
      }
    }
  }
 
  await Promise.all(Array.from({ length: Math.min(concurrencyLimit, items.length) }, worker));
  return results;
}

Because JavaScript is single-threaded, nextIndex++ is atomic — no two workers can claim the same index. At any moment, exactly concurrencyLimit tasks are in flight. As soon as one finishes, the worker immediately picks up the next item. Memory usage stays flat regardless of input size, because you never allocate more than N unresolved promises.

The settled-result return type is intentional. Rather than letting one failure abort the entire batch (as Promise.all would), every item gets an independent outcome. Your caller decides what to do with failures.

Adding Retry with Exponential Backoff

Transient failures — rate limits, DNS hiccups, brief upstream outages — are common in production. Wrap your core operation with a retry decorator before feeding it to the pool:

export function withRetry<Args extends unknown[], R>(
  fn: (...args: Args) => Promise<R>,
  { maxRetries, initialDelayMs, factor }: { maxRetries: number; initialDelayMs: number; factor: number }
): (...args: Args) => Promise<R> {
  return async (...args) => {
    for (let attempt = 0; attempt <= maxRetries; attempt++) {
      try {
        return await fn(...args);
      } catch (error) {
        if (attempt === maxRetries) throw error;
        const delay = Math.random() * initialDelayMs * Math.pow(factor, attempt);
        await new Promise(resolve => setTimeout(resolve, delay));
      }
    }
    throw new Error('unreachable');
  };
}

The jitter (Math.random() * baseDelay) is important. Without it, all workers retry at the same moment after a rate limit response, creating a "thundering herd" that immediately hits the same limit again.

Putting It Together

const sendWithRetry = withRetry(sendTelemetryRaw, { maxRetries: 3, initialDelayMs: 100, factor: 2 });
 
const results = await concurrentMapSettled(payloads, 10, payload => sendWithRetry(payload));
 
const failures = results.filter(r => r.status === 'rejected');
if (failures.length > 0) {
  // Send to dead-letter queue or alert
}

Ten workers in flight, three retries each with jittered backoff, and no single failure aborts the batch. That's the pattern for any large-scale ingestion pipeline.

One Caveat: Memory at Scale

concurrentMapSettled allocates an output array the same size as the input. For millions of items, keeping all results in memory will eventually exhaust the heap. At that scale, replace the in-memory array with a Node.js stream or an async generator pipeline that processes results chunk-by-chunk as they complete, rather than holding everything until the batch is done.