Featured in this episode of Chaos Lever
First we had the great S3 outage of 2017. Then the DNS and network storm of 2021. Both severely impacted operations in the us-east-1 region, aka the default region that most people deploy to.
And each time, cloud pundits hollered from the mountain tops for companies to migrate off of us-east-1 or at least stretch their applications to be fault-tolerant to regional outages. Heck, even AWS itself promised to update their Customer Service app to be more resilient. And yet…
On June 13th, AWS started experiencing high error rates from their Lambda service in us-east-1. Other services became impacted due to their reliance on Lambda, and soon websites, applications, and services for customers started to fail.
For instance, I got a message from Zapier that some of my Zaps might be delayed or wouldn’t be running during the outage. AWS’ Customer Service app also experienced delays and outages, indicating that they still have some cleanup to do.
In the case of Zapier, I don’t really care that my automations had to wait an extra hour or two to fire, but if you’re providing a time-sensitive service to customers, it might be time to check and make sure you’re not just using us-east-1. Like, for real this time.