toggle

AAPL Stock: 114.64 ( -3.8 )

Printed from http://www.macnn.com

Amazon EC2 data center failure affects Netflix, Instagram

updated 09:13 pm EDT, Sat June 30, 2012

Storms in North Virginia takes out Amazon data center

Netflix, Instagram, and other web services were hit by an Amazon Web Services outage. Historic thunderstorms in Northern Virginia caused power outages for Amazon data center in the region, forcing services depending on EC2 from it to be severely affected, with Netflix in particular suffering considerable amounts of downtime.

At the time of writing, the Amazon Web Services status page shows the power issues for its Elastic Compute Cloud as Resolved after lasting for 20 hours. Amazon CloudSearch, Elastic Beanstalk, and Relational Database Services in Northern Virginia were also affected, forcing the same services offered in other data centers around the world to pick up the extra work.

Power providers to the area warn of sporadic outages in the vicinity of the server farm over the next several days, as power is gradually restored to the nearly 500,000 who lost it during the storm. Full restoration of the power grid in the area could take upwards of a week.

MacNN has spoken with an electrician that works with the service. An electrical breaker with a safety interlock fused shut during the storm. This breaker being stuck prevented the diesel generator's breaker from closing, as only one of the breakers can be shut at any given time, so no backup power was available to the AWS service.

Amazon suffered a similar outage in August last year, again affecting the Northern Virginia data center. More services were affected at that point, which caused Foursquare, Reddit and Quora to work on spreading their services across multiple locations and have a more robust backup-plan for data center failures. [via TechCrunch]




by MacNN Staff

toggle

Comments

  1. msuper69

    Joined: Dec 1969

    -8

    UPS anyone?

    They do make data center grade UPSs.

    This just shows that Amazon, et. al. aren't serious about keeping their systems up during storms.
    Bad management.

  1. The Vicar

    Joined: Dec 1969

    +2

    The Cloud

    Everyone with actual experience in the IT industry has been saying since the beginning that "The Cloud" is a dumb idea with this being one of the possible ways it can go wrong. This is only one of the first times it happens. Eventually, people will get the message that the whole idea is just plain dumb.

    I just wonder whether that will happen before or after the "Cloud security failure" scenario happens in real life. That one will be... interesting.

  1. neocyberdude

    Joined: Dec 1969

    +10

    UPS/Generators

    Yes, they have VERY large UPSs with entire rooms full of batteries, but just like your home UPS, they are designed to ride through short power outages (15-20 minutes or so). After that they have diesel generators that can carry the load for another few hours, but this was a very long power outage. Theoretically they could have brought in truckloads of diesel fuel, but I imagine they were counting on the other data centers to pick up the traffic load. It didn't work, apparently.

  1. fizzy

    Joined: Dec 1969

    +5

    regional storm

    This was a very large, and unexpectedly violent storm. The outages were not limited to N. Va. Power and networking were disrupted in the whole Mid-Atlantic area. Amazon could have had perfect power backup and still been disrupted, as could any business running their own servers anywhere in that area.

  1. chas_m

    Joined:

    +5

    good point, fizzy, but

    really these companies should all spread their data centres around a bit more. Like, you know, Apple.

  1. Grendelmon

    Joined: Dec 1969

    +1

    You all can speculate

    ...as much as you want about how "poor" Amazon manages it's infrastructure, but unless you work there, you have no idea what happened. I have yet to work with a (large) company that does NOT take it's disaster recovery seriously. And for a company like Amazon, even to resell their data centers to other providers, has got to have a serious plan in place. The storms that occurred were historic, Let's not forget about the fact that a dozen people were killed across multiple states, which nobody apparently finds important aside the fact that "Instacrap" went down. And NetFlix. Oh damn, what will the world do for a day or two.

  1. AngryFanBoy

    Joined: Dec 1969

    0

    Netflix?!?

    Oh horrors! I won't be able to stream Archer for several hours :(

    What to do???

Login Here

Not a member of the MacNN forums? Register now for free.

toggle

Network Headlines

Follow us on Facebook

toggle

Most Popular

Advertisement

Recent Reviews

HP 14-x030nr 14-inch Chromebook

If you're like us, chances are you've come to realize that you need the ability to access the Internet on the go. Also, you've prob ...

15-inch MacBook Pro with Force Touch

Apple's 15-inch Retina MacBook Pro continues to be a popular notebook with professional users and prosumers looking for the ultimate ...

Typo keyboard for iPad

Following numerous legal shenanigans between Typo -- a company founded in part by Ryan Seacrest -- and the clear object of his physica ...

toggle

Most Commented