Featured in this episode of Chaos Lever
The Google Cloud’s europe-west9-a zone suffered a significant amount of damage back in late April. While the Goog has been evasive on what the exact situation was, terms like “water intrusion” and “fire incident” have been thrown around in press releases.
French forums discussing the event believe there was a cooling pump that failed, leading to a leak, which ignited a fire in the battery room. If you’re keeping score at home, none of those words are good, near computers or just like, in general. Whatever the specifics were, the intrusion incident was severe enough that the zone still has not recovered some 3 weeks later.
Just a reminder that while these kinds of incidents are exceedingly rare, they do happen in on-prem datacenters as well as in cloud datacenters. But it’s not a big deal, all of your production workloads are designed to survive a zone failure, right? …right?