Designing reliable large-file uploads

Large uploads fail at browser, proxy, network and storage boundaries. A resilient design separates transfer from business acceptance and allows an interrupted user to continue without restarting the entire file.

Set an explicit upload contract

Define maximum size, permitted media, expected duration, retention of incomplete transfers and who may create an upload. Return limits before transfer and repeat authoritative validation after receipt.

Transfer directly to durable storage

Issue short-lived, scoped upload credentials or signed URLs so data can reach object storage without occupying the application request pool. Restrict object key, size and content conditions where the storage service supports them.

Use resumable parts

Split files into numbered parts with an upload identifier, record completed checksums and allow retries for individual parts. Complete the object only when the declared part list and total size match.

Validate asynchronously

Treat upload completion as quarantined, then scan, inspect media type, calculate a final checksum and apply business rules in a worker. Never trust the browser filename or Content-Type as the security decision.

Operate cleanup and user recovery

Expire abandoned sessions and unreferenced objects through a monitored lifecycle. Show progress, reconnection and processing state to the user, and make final submission idempotent if the browser repeats it.