File Upload System

The Problem

Design an uploader that accepts large files - a two gigabyte video, a folder of raw images - shows meaningful progress, survives a dropped connection, and can be cancelled cleanly.

The naive version is a <form> with enctype="multipart/form-data". It works for a profile picture and fails for anything substantial, for three separate reasons: one long request is an all-or-nothing bet that grows more fragile the longer it runs, your API servers should not be carrying gigabytes of user data, and progress reporting on a single stream is not resumable state. All three are architectural rather than cosmetic.

Requirements

Functional

• Upload files up to several gigabytes with visible progress.
• Resume an interrupted upload without restarting it, including across a page reload.
• Cancel an in-progress upload and clean up server-side state.
• Validate type and size before transferring bytes.
• Multiple concurrent uploads with independent and aggregate progress.

Non-functional

• File bytes never traverse the application server.
• A failed chunk retries without affecting the rest of the transfer.
• Available bandwidth is used without saturating the connection or the main thread.
• Uploaded content is validated server-side regardless of what the client claimed.

Direct-to-Storage with Presigned URLs

Start here, because it determines everything downstream.

Proxying means the browser posts to your API, which streams the bytes onward to storage. Simple, and every byte flows through your compute. A hundred concurrent one-gigabyte uploads is a hundred gigabytes through servers whose real job is a few kilobytes of JSON - holding connections and memory buffers open for minutes, and forcing you to scale compute for bandwidth.

Direct-to-storage means your API issues a presigned URL: a time-limited, operation-scoped credential authorising exactly one write to exactly one object key. The browser uploads straight to storage. Your API issues credentials and records outcomes.

Diagram
100%
flowchart TB
    B["Browser"]

    subgraph API["Your API - small, stateless, no bytes"]
        AUTH["1. authorise the user<br/>check quota and permissions"]
        SIGN["2. create upload session<br/>choose object key<br/>sign per-chunk URLs"]
        DONE["5. finalise<br/>verify + record in DB"]
    end

    subgraph STORE["Object storage - built for bytes"]
        S3["multipart upload<br/>part 1..N"]
        OBJ["assembled object"]
    end

    B -->|"POST /uploads<br/>name, size, type, checksum"| AUTH
    AUTH --> SIGN
    SIGN -->|"uploadId, key,<br/>presigned URL per part"| B
    B -->|"3. PUT part 1..N directly<br/>(bytes never touch the API)"| S3
    S3 -->|"ETag per part"| B
    B -->|"4. POST /uploads/:id/complete<br/>parts + ETags"| DONE
    DONE -->|"CompleteMultipartUpload"| S3
    S3 --> OBJ
    DONE -->|"enqueue scan + transcode"| Q["background worker"]

    style API fill:#1e3a5f,stroke:#3b82f6
    style STORE fill:#1e3f2d,stroke:#22c55e
    style Q fill:#3f2d1e,stroke:#f59e0b
Zoom or open fullscreen for a closer look
visualized by

The signing request is where all the authorisation happens, and the signature is where the constraints live:

// Server-side. The client never picks the key and never sees a long-lived credential.
export async function createUploadSession(input: CreateUploadInput, userId: string) {
  const parsed = CreateUploadSchema.safeParse(input);
  if (!parsed.success) return { error: 'Invalid upload request' };

  const { fileName, fileSize, contentType } = parsed.data;

  if (fileSize > MAX_UPLOAD_BYTES) return { error: 'File too large' };
  if (!ALLOWED_CONTENT_TYPES.includes(contentType)) {
    return { error: 'Unsupported file type' };
  }
  if (await isOverQuota(userId, fileSize)) return { error: 'Quota exceeded' };

  // The server owns the key. A client-supplied key is a path traversal and an
  // overwrite-anything vulnerability in one.
  const key = `uploads/${userId}/${crypto.randomUUID()}/${sanitiseFileName(fileName)}`;

  const { uploadId } = await storage.createMultipartUpload({ key, contentType });
  const partCount = Math.ceil(fileSize / CHUNK_SIZE_BYTES);

  const urls = await Promise.all(
    Array.from({ length: partCount }, (_, index) =>
      storage.signUploadPart({
        key,
        uploadId,
        partNumber: index + 1,
        expiresInSeconds: 60 * 60,
        // Constraints enforced by the signature itself, not by client goodwill.
        contentLengthRange: [1, CHUNK_SIZE_BYTES],
      }),
    ),
  );

  await db.uploadSession.create({
    data: { id: uploadId, key, userId, fileSize, contentType, status: 'PENDING' },
  });

  return { data: { uploadId, key, urls, chunkSize: CHUNK_SIZE_BYTES } };
}

Four things that must be true, each corresponding to a real vulnerability:

1. The server chooses the object key. A client-supplied key means path traversal and overwriting other users' objects.
2. Constraints are baked into the signature. A content-length range and a fixed content type are enforced by storage at write time, so a client that lies about the file cannot exceed them.
3. Short expiry. A leaked URL is a write credential; an hour bounds the damage. Long uploads re-sign rather than getting a longer window.
4. Nothing is trusted until verified. The object exists after upload but is not yet valid - see finalisation below.

This is the credential-scoping principle from Security Architecture in its most concrete form: a capability narrow enough that possessing it grants only the one operation you intended.

Proxying remains right in two cases: when bytes must be inspected or transformed in-flight (a compliance requirement to scan before storing), or when your storage provider offers no presigning. Otherwise direct upload is the default at any scale.

Chunking

Split the file client-side and upload the pieces. File extends Blob, and slice is cheap - it creates a view, not a copy, so slicing a two gigabyte file costs nothing in memory.

const CHUNK_SIZE_BYTES = 8 * 1024 * 1024; // 8 MB
const MAX_PARALLEL_CHUNKS = 3;

type ChunkState = {
  index: number;
  start: number;
  end: number;
  size: number;
  status: 'pending' | 'uploading' | 'done' | 'failed';
  bytesSent: number;
  attempts: number;
  etag?: string;
};

function planChunks(file: File): ChunkState[] {
  const chunks: ChunkState[] = [];

  for (let start = 0; start < file.size; start += CHUNK_SIZE_BYTES) {
    const end = Math.min(start + CHUNK_SIZE_BYTES, file.size);
    chunks.push({
      index: chunks.length,
      start,
      end,
      size: end - start,
      status: 'pending',
      bytesSent: 0,
      attempts: 0,
    });
  }

  return chunks;
}

Chunk size is a real tradeoff. Small chunks (1MB) mean cheap retries and smooth progress, but per-request overhead dominates and a 2GB file becomes 2,000 requests. Large chunks (50MB) amortise overhead but make a retry expensive and progress coarse. 5-10MB is the usual range; S3's multipart minimum of 5MB for non-final parts effectively sets the floor. Adapting to measured throughput - smaller chunks on a slow link where a large retry is punishing - is a refinement worth mentioning.

Parallelism is a second tradeoff. Sending chunks concurrently uses available bandwidth; too many saturates the uplink, making every chunk slow and progress erratic, and competes with the rest of the application for the browser's ~6 connections per origin. Three to four in flight is the practical sweet spot.

async function runUpload(session: UploadSession, chunks: ChunkState[]) {
  const queue = chunks.filter((chunk) => chunk.status !== 'done');
  let cursor = 0;

  async function worker() {
    while (cursor < queue.length && !session.aborted) {
      const chunk = queue[cursor];
      cursor += 1;
      await uploadChunkWithRetry(session, chunk);
    }
  }

  // A fixed pool of workers pulling from one queue. Simpler and more predictable
  // than batching, which stalls on the slowest chunk in each batch.
  await Promise.all(
    Array.from({ length: MAX_PARALLEL_CHUNKS }, () => worker()),
  );
}

A worker pool beats batching because a batch waits for its slowest member before starting the next group, leaving connections idle. Workers keep all lanes busy.

Per-chunk upload with retry and progress

Progress events are the one thing fetch still cannot do for uploads, so this is the rare case where XMLHttpRequest remains correct:

function uploadChunk(
  url: string,
  body: Blob,
  onProgress: (bytesSent: number) => void,
  signal: AbortSignal,
): Promise<string> {
  return new Promise((resolve, reject) => {
    const request = new XMLHttpRequest();
    request.open('PUT', url);

    // fetch() has no upload progress. This is why XHR is still used here.
    request.upload.onprogress = (event) => {
      if (event.lengthComputable) onProgress(event.loaded);
    };

    request.onload = () => {
      if (request.status >= 200 && request.status < 300) {
        // Storage returns the part's ETag; completion needs all of them.
        resolve(request.getResponseHeader('ETag') ?? '');
      } else {
        reject(new Error(`Chunk upload failed: ${request.status}`));
      }
    };

    request.onerror = () => reject(new Error('Network error during chunk upload'));
    request.ontimeout = () => reject(new Error('Chunk upload timed out'));

    signal.addEventListener('abort', () => request.abort(), { once: true });
    request.send(body);
  });
}

const MAX_CHUNK_ATTEMPTS = 4;

async function uploadChunkWithRetry(session: UploadSession, chunk: ChunkState) {
  chunk.status = 'uploading';

  for (let attempt = 1; attempt <= MAX_CHUNK_ATTEMPTS; attempt += 1) {
    chunk.attempts = attempt;

    try {
      const body = session.file.slice(chunk.start, chunk.end);
      chunk.etag = await uploadChunk(
        session.urls[chunk.index],
        body,
        (bytesSent) => {
          chunk.bytesSent = bytesSent;
          session.onProgress();
        },
        session.abortController.signal,
      );

      chunk.status = 'done';
      chunk.bytesSent = chunk.size;
      await persistProgress(session); // survive a reload
      return;
    } catch (error) {
      if (session.aborted) throw error;

      // Reset for the retry, but never below what we have already reported -
      // progress that goes backwards destroys the user's trust in the number.
      chunk.bytesSent = 0;

      if (attempt === MAX_CHUNK_ATTEMPTS) {
        chunk.status = 'failed';
        throw error;
      }

      const backoff = Math.min(1000 * 2 ** (attempt - 1), 15_000);
      await sleep(backoff + Math.random() * 500); // jitter
    }
  }
}

Because each chunk is an independent idempotent PUT to a specific part number, a retry is safe: re-uploading part 7 replaces part 7. That property is what makes chunked uploads robust, and it is worth naming in an interview - it is the same idempotency argument that makes retry safe anywhere.

Resumable Uploads

A resumable upload needs both sides to agree on which parts already exist, which requires an identity for the transfer that outlives the connection - and, ideally, the browser session.

Diagram
100%
sequenceDiagram
    participant U as User
    participant C as Client
    participant IDB as IndexedDB
    participant API as Your API
    participant S as Object storage

    U->>C: selects video.mp4 (1.2 GB)
    C->>C: derive uploadKey = hash(first 1MB + size + lastModified)
    C->>API: POST /uploads (uploadKey, size, type)
    API-->>C: uploadId, key, presigned part URLs
    C->>IDB: persist {uploadKey, uploadId, chunk states}

    C->>S: PUT part 1
    S-->>C: 200, ETag
    C->>IDB: mark part 1 done
    C->>S: PUT part 2
    S-->>C: 200, ETag
    C->>IDB: mark part 2 done

    C->>S: PUT part 3
    Note over C,S: connection drops mid-part
    S--xC: network error
    C->>C: backoff 1s, retry part 3
    C->>S: PUT part 3 (full part re-sent)
    S-->>C: 200, ETag

    Note over U,S: user closes the tab - memory is gone,<br/>the File handle is gone, IndexedDB is not

    U->>C: returns, re-selects video.mp4
    C->>C: derive same uploadKey
    C->>IDB: found session - parts 1-3 done
    C->>API: GET /uploads/:id/parts (verify against storage)
    API->>S: ListParts
    S-->>API: parts 1,2,3 with ETags
    API-->>C: resume from part 4
    C->>S: PUT part 4..N
    C->>API: POST /uploads/:id/complete (all ETags)
    API->>S: CompleteMultipartUpload
    API-->>C: 201 - object ready
Zoom or open fullscreen for a closer look
visualized by

Two subtleties in that flow are the whole of resumability.

The identity must be derivable from the file, not assigned by the server. After a reload the client has to recognise "this is the same file I was uploading", and the only stable input is the file itself:

async function deriveUploadKey(file: File): Promise<string> {
  // Hashing 1.2 GB to identify a file is wasteful. The first megabyte plus
  // size and mtime is enough to distinguish files in practice.
  const head = await file.slice(0, 1024 * 1024).arrayBuffer();
  const digest = await crypto.subtle.digest('SHA-256', head);
  const hex = Array.from(new Uint8Array(digest))
    .map((byte) => byte.toString(16).padStart(2, '0'))
    .join('');

  return `${hex}-${file.size}-${file.lastModified}`;
}

The File object cannot be persisted. It is a handle to data the page does not own, and it dies with the page. So a resumed upload after a reload requires the user to re-select the file - at which point the derived key matches the stored session and the transfer continues. Presenting this honestly in the UI ("Resume upload - reselect video.mp4") is better than pretending it resumes automatically. The File System Access API changes this for browsers that support it, since a persisted handle can be re-authorised, but re-selection remains the portable path.

Progress must live in IndexedDB, not localStorage - it is written frequently, it can exceed the 5MB string quota across concurrent uploads, and localStorage is synchronous and blocks the main thread. This is the same durable-queue reasoning as Offline and PWA Architecture, and an upload is one of the clearest cases for it: the thing being lost is minutes of the user's time and bandwidth.

Verify against storage on resume, not just against local state. Multipart uploads expire, a lifecycle policy may have swept the parts, or another device may have completed the same session. ListParts is the authority; local state is a hint.

Validation

Two layers doing different jobs, and conflating them is the mistake.

Client-side, for feedback speed:

const MAX_UPLOAD_BYTES = 2 * 1024 * 1024 * 1024;

const MAGIC_NUMBERS: { bytes: number[]; type: string }[] = [
  { bytes: [0xff, 0xd8, 0xff], type: 'image/jpeg' },
  { bytes: [0x89, 0x50, 0x4e, 0x47], type: 'image/png' },
  { bytes: [0x25, 0x50, 0x44, 0x46], type: 'application/pdf' },
];

async function sniffContentType(file: File): Promise<string | null> {
  const header = new Uint8Array(await file.slice(0, 16).arrayBuffer());

  for (const signature of MAGIC_NUMBERS) {
    if (signature.bytes.every((byte, index) => header[index] === byte)) {
      return signature.type;
    }
  }

  return null;
}

async function validateBeforeUpload(file: File) {
  if (file.size === 0) return { error: 'File is empty' };
  if (file.size > MAX_UPLOAD_BYTES) {
    return { error: `File exceeds ${formatBytes(MAX_UPLOAD_BYTES)}` };
  }

  // file.type comes from the OS by extension and is trivially wrong or absent.
  // Reading the header is cheaper than uploading and being rejected.
  const sniffed = await sniffContentType(file);
  if (!sniffed || !ALLOWED_CONTENT_TYPES.includes(sniffed)) {
    return { error: 'Unsupported file type' };
  }

  return { data: { contentType: sniffed } };
}

The value here is purely that the user learns in 50ms rather than after a five-minute transfer. file.type is derived from the extension by the operating system, so a renamed .exe reports whatever you renamed it to; reading magic numbers is strictly better and costs one small slice.

Server-side, for actual enforcement. Every client check runs in an environment the user controls. Enforcement lives in three places: the presigned URL's constraints (length range, content type, server-chosen key), an object-level check after upload in the finalise step, and asynchronous verification - malware scan, image re-encode, media probe - before the object is exposed to anyone. Until that completes, the object is PENDING and unreadable by other users.

The rule to state plainly: client validation is a fast path; the signature and post-upload verification are the security boundary.

Progress Aggregation

Progress must be trustworthy, which means monotonic and roughly linear in time.

type UploadProgress = {
  bytesSent: number;
  totalBytes: number;
  percent: number;
  bytesPerSecond: number;
  secondsRemaining: number | null;
};

class ProgressTracker {
  private samples: { at: number; bytes: number }[] = [];
  private reportedFloor = 0;

  constructor(private totalBytes: number) {}

  compute(chunks: ChunkState[], now: number): UploadProgress {
    const raw = chunks.reduce(
      (sum, chunk) => sum + (chunk.status === 'done' ? chunk.size : chunk.bytesSent),
      0,
    );

    // A retried chunk resets its bytesSent to zero. Never let the number the
    // user is watching go backwards because of an internal retry.
    const bytesSent = Math.max(raw, this.reportedFloor);
    this.reportedFloor = bytesSent;

    // Rolling window, not cumulative average: a cumulative rate reacts far too
    // slowly to a network change and produces the estimate stuck at "2 minutes".
    this.samples.push({ at: now, bytes: bytesSent });
    this.samples = this.samples.filter((sample) => now - sample.at < 5_000);

    const oldest = this.samples[0];
    const elapsedSeconds = (now - oldest.at) / 1000;
    const bytesPerSecond =
      elapsedSeconds > 0.5 ? (bytesSent - oldest.bytes) / elapsedSeconds : 0;

    const remaining = this.totalBytes - bytesSent;

    return {
      bytesSent,
      totalBytes: this.totalBytes,
      percent: Math.min(100, (bytesSent / this.totalBytes) * 100),
      bytesPerSecond,
      secondsRemaining:
        bytesPerSecond > 0 ? Math.ceil(remaining / bytesPerSecond) : null,
    };
  }
}

Four properties that make a progress bar believable:

• Aggregate bytes, not completed chunks. Counting chunks makes progress jump in 8MB steps and appear frozen while three chunks are mid-flight.
• Monotonic. A retry must not subtract bytes already reported.
• Rolling throughput. A five-second window tracks reality; a cumulative average lags badly.
• Honest about unknowns. secondsRemaining: null before there is enough data, rendered as "calculating…" rather than a fabricated number.

For multiple files, the aggregate bar should weight by size rather than count files - four small files and one large one is not "20% done" after the first.

Throttle progress renders to about 10 per second. onprogress can fire far more often than that, and re-rendering per event burns main-thread time that the upload's own encryption and I/O also need.

Cancellation and Cleanup

Cancelling has a client half and a server half, and skipping the second one costs money.

async function cancelUpload(session: UploadSession) {
  session.aborted = true;
  session.abortController.abort(); // stops in-flight chunk PUTs

  await api.abortUpload(session.uploadId); // AbortMultipartUpload
  await clearPersistedProgress(session.uploadKey);
}

Uploaded parts of an incomplete multipart upload remain in storage and remain billable until the upload is aborted or expired. Two safeguards: call abort explicitly on cancel, and set a storage lifecycle rule that deletes incomplete multipart uploads after a few days - because a browser that closes mid-upload never tells you anything.

The beforeunload guard is worth adding and worth understanding the limits of:

useEffect(() => {
  if (!hasActiveUploads) return;

  const warn = (event: BeforeUnloadEvent) => {
    event.preventDefault();
    // Browsers show their own generic message; custom text is ignored.
    event.returnValue = '';
  };

  window.addEventListener('beforeunload', warn);
  return () => window.removeEventListener('beforeunload', warn);
}, [hasActiveUploads]);

It is a prompt, not a guarantee - the user can dismiss it, and a crash bypasses it entirely. Persisted progress is what actually protects the transfer.

The Drop Zone

Accepting files dragged from the desktop is the one place native HTML5 drag and drop is correct, because it is the only API that receives an OS-level drag. This is the exception noted in Drag and Drop.

<div
  onDragOver={(event) => {
    event.preventDefault(); // without this, drop never fires
    setIsOver(true);
  }}
  onDragLeave={() => setIsOver(false)}
  onDrop={(event) => {
    event.preventDefault();
    setIsOver(false);
    // Prefer items over files: it exposes directory entries too.
    void addFiles(Array.from(event.dataTransfer.files));
  }}
  className={cn('rounded-xl border-2 border-dashed p-8', isOver && 'border-yellow-400')}>
  <p>Drag files here, or</p>
  {/* The visible control is a real input, so keyboard and screen reader users
      have a first-class path rather than an inaccessible drop target. */}
  <label className='cursor-pointer underline'>
    browse
    <input
      type='file'
      multiple
      accept={ALLOWED_CONTENT_TYPES.join(',')}
      className='sr-only'
      onChange={(event) => void addFiles(Array.from(event.target.files ?? []))}
    />
  </label>
</div>

The <input type="file"> is not a fallback - it is the primary control, and the drop zone is an enhancement. A drop-only uploader is unusable without a pointer.

Also prevent the document default, or dropping a file outside the zone navigates the tab away from your app and loses everything in progress:

useEffect(() => {
  const swallow = (event: DragEvent) => event.preventDefault();
  window.addEventListener('dragover', swallow);
  window.addEventListener('drop', swallow);
  return () => {
    window.removeEventListener('dragover', swallow);
    window.removeEventListener('drop', swallow);
  };
}, []);

Common Interview Follow-Up Questions

"How do you know the upload succeeded if the browser closes right after the last chunk?" You do not, and the design must not depend on the browser to tell you. The completion call is a client-initiated step that can be lost, so treat it as an optimisation rather than the source of truth: storage should notify your backend directly through an event notification when an object is created, and the finalise endpoint must be idempotent so a duplicate call from a retrying client is harmless. For extra safety, a periodic reconciliation job lists multipart uploads whose parts are all present but which were never completed and finishes or aborts them. This is the same lost-acknowledgement problem that makes client-generated ids necessary in Real-Time Feed.

"Uploading 500 small files. Does chunking help?" No - it makes things worse. Files below the chunk size should upload as single PUT requests, and the bottleneck becomes request overhead and connection concurrency rather than bandwidth. Batch the signing request so one API call returns 500 URLs instead of making 500 round trips just to get credentials, run a worker pool of 4-6 uploads, and consider client-side zipping when the files are logically one unit, since one 50MB archive beats 500 requests. The progress model changes too: per-file rows collapse into a summary ("312 of 500 uploaded, 4 failed") with the failures individually retryable.

"How do you upload from a Service Worker so it survives navigation?" The Background Fetch API is the intended answer - it hands a fetch to the browser, which continues it outside the page's lifetime, shows OS-level progress, and wakes the Service Worker on completion. Support is limited, so the portable approach is what is described above: durable progress in IndexedDB plus resume on return. A Service Worker can also help by owning the upload loop so that navigating between routes in a single-page app does not tear it down, which is a smaller but real win. The lifecycle constraints are in Offline and PWA Architecture.

"The user uploads a 4GB video and the tab's memory spikes. Why?" Something is reading the file into memory instead of streaming a view of it. file.slice() is a view and costs nothing; await chunk.arrayBuffer() materialises the bytes, and doing that for several parallel chunks means hundreds of megabytes resident. Hash only a small prefix rather than the whole file, pass the Blob straight to send() rather than converting it, and never build a base64 string of file contents - base64 inflates by a third and is a common cause of tab crashes on large files.

"What do you instrument?" Success rate by file size bucket and by connection type, which is where the real failures hide - a 98% overall success rate can conceal 60% for files over 1GB on cellular. Then chunk retry rate, time-to-first-byte on the signing call, end-to-end duration per megabyte, resume rate (how often users actually come back), and abandonment by percent complete, which tells you whether people are giving up at a specific point. Cancellations and expiries also want tracking because they cost storage. All of it must be reported from the client, since the API never sees the bytes - the argument for client-side telemetry made in Observability.

Tradeoffs Table

Where This Applies

File upload is where three foundations become concrete at once. The request lifecycle from Networking and Data Fetching shows up as cancellation, retry with backoff and per-chunk idempotency. The presigned URL is the narrowest possible capability, which is exactly the credential-scoping argument in Security Architecture - together with the rule that client validation is feedback and server verification is enforcement. And resumability across a reload is the durable-queue pattern from Offline and PWA Architecture, applied to bytes rather than to mutations.

Within this track, the drop zone is the one legitimate use of native HTML5 drag and drop discussed in Drag and Drop. The retry-with-backoff and lost-acknowledgement problems are the same ones solved in Real-Time Feed. And an uploader is usually the front half of an image gallery, where the objects it produces become the responsive variants that page serves.