Skip to main content

What happened to Amazon yesterday?

Yesterday was an interesting day for many (I mean many) sites around the world which rely on Amazon servers.

Amazon EC2 (Elastic Compute Cloud), an amazing implementation of Cloud Computing had a major network failure yesterday that triggered a re-mirror of all the Elastic Block Storage (EBS) volumes in the North Virginia data centre.

This re-mirror combined with a large number of customers trying to fire up new instances to recover from the network failure caused Amazon to hit a storage capacity problem.

 

image
Basically they didn’t have enough disk space to support the re-mirror and as the thousands upon thousands of customers they have tried to fire up new replacement images the problem compounded.

Amazon is still experiencing these issues and is updating the current status at http://status.aws.amazon.com/rss/EC2.rss and http://status.aws.amazon.com/

Many big sites and their services are still down. Some of them are -

About.me

Quora.com

Reddit.com

StarNews.in

RepositoryHosting.com

Foursquare.com

HootSuite.com

See http://ec2disabled.com for the complete list.

What companies can try in these scenarios?

  1. They should have strict disaster recovery process which can make the system up in not less than 30 minutes
  2. They should try out the actual worst scenarios from their so called hourly (or minutely!) backups and thorough recovery process
  3. As soon as Amazon admitted that they had severe performance issues in the status page, you should start out disaster recovery process, in this case, move everything to a new availability zone to have everything up and running with full data integrity

Do you really believe that Amazon running out of disk space? But those are the details that Amazon published as of now. We may have to wait for full post mortem report.

Labels:

Comments

Post a Comment

Popular posts from this blog

Microsoft Officially Confirmed New Windows Logo

Microsoft changed its Windows Logo, an inspired design from its new Metro interface. A day after Apple unveiled its next operating system Mountain Lion, Microsoft announced in an official blog confirmation that it has redesigned the Windows Logo. This new logo is truly inspired from its well known mobile operating system Windows Phone.         The new design is the biggest change from its classic 90s logo. In an official post , Microsoft’s Principal Director of User Experience for Windows confirmed that the new logo carries Metro principle of being ‘Authentically Digital’. “ It’s a window… not a flag …., “your name is Windows. Why are you a flag?” In some ways you can trace the evolution of the Windows logo in parallel with the advancements of the technology used to create logos. From the simple two color version in Windows 1.0 to the intricate and detailed renderings in Windows Vista and Windows 7, each change makes sense in the context i...
Post a Comment
Read more

Breaking: Microsoft India Store Has Been Hacked. Passwords In Plain Text Format Exposed [Update]

Well, this is huge! Microsoft’s India online store website has been hacked and all user information which was stored in plain text format without any encryption has also been exposed. It looks like the hacker team got remote access to the whole web server. The main reasons behind this hack are still unknown but from the homepage replaced information, its been predicted that a non well known hacker organization EvilShadow is behind all this invasion.     Microsoft pulled out the store page right now and might be working on this to get back the site. The unacceptable thing in all this story is that the software giant stored all the user information including passwords in the database in plain text without any encryption for such online shopping site. Update (as of 11:45 PM Central Time, 2/12/12) – The Microsoft Store India is still currently unavailable to users. To patch up things, Microsoft is asking customers to change their passwords immediately. Full email me...
7 comments
Read more

Windows Phone 7.5 Tango Officially Renamed As ‘Refresh’

The next version of Microsoft’s Windows Phone operating system after Mango 7.1 is officially renamed as ‘Windows Phone 7.5 Refresh’ – according to the head of Windows Phone division for Microsoft Italy.     According to Italian version of the post , 7.5 Refresh update will mostly be an update to the minimum specs of the devices it’ll be able to run (minimum RAM requirements is dropped from 512 MB to 256 MB). Other updates which may include in this release are better media messaging, location awareness icon, export and manage contacts to SIM card. The next major update after ‘Refresh’ is called Windows Phone Apollo (probably Windows Phone 8) could certainly be an exciting release from Microsoft. This Apollo update may have BitLocker kind of support on mobile devices, multi-core support. [ via ] [ Image ]
Post a Comment
Read more