When your cloud suppliers is out, you’re out too. For Amazon the disaster was a “Performance issue” according to their health check dashboard. We have a different story to tell, for us this meant a 100% outage, a “Service disruption”. Yesterday we booted up the primary functions and today we’ve recovered most of the Mobile Documents system, wrecked by Amazon’s backup system running amok. This is what happened.
19:41 CET, Sunday August 7, 2011, a lightning strikes out a transformer in Dublin, Ireland. The lightning sparks a fire and explosion, and propagates into Amazon’s data center where its backup generators fail, that normally kicks in automatically. An Amazon cloud power outage is a fact. And out goes the Mobile Documents service as well. You could expect that a top-notch modern data center should be able to withstand a thunder storm. No problem you may think, the center will soon get power and Amazon can restart their instances.
