S3 Is Infinitely Scalable. Your Code Is Not.

·1 min read·

Amazon S3 can handle anything you throw at it. The code that talks to S3 often cannot.

S3 is one of the most reliable infrastructure primitives AWS offers. Eleven nines of durability. Effectively unlimited scale.

And yet S3-related incidents are surprisingly common. Not because S3 failed — because the integration code was written without considering failure modes.

The patterns that cause incidents: synchronous uploads in your web request cycle, missing exponential backoff on retries, presigned URLs that expire too soon.

The correct patterns: upload to S3 asynchronously via a queue. Implement exponential backoff with jitter. Handle partial upload failures with multipart upload verification.

S3 can scale to any workload. Your integration layer determines whether that scale is an asset or a liability.

— Dick Bassey | DevDick | 2023

Share this article
← Back to all writing