Re:Facebook is down
by mrmaddog on Tuesday June 18, 2013 @09:16PM
(#5903152)
One of the things I remember from an internship there a few years ago, Facebook does its main "push" of new code to its server farms on Tuesdays around 5pm (i.e. around this time). It would be interesting to read a postmortem on what brought down the most visited site on the world—was it an image staging issue? A latent bug in the product? A badly configured setting? A malicious attack? A network problem? Hopefully Facebook Eng will do a write-up on the root cause!
"Every week, usually on Sunday evenings, we stage our weekly push, which happens on Tuesday evenings, on the beta tier. This provides ourselves and developers 48 hours to test and find bugs before the build hits production. After Tuesday, while the beta tier is continuously updated with hotfix patches which are queued up for the daily hotfix release, the production tier is updated with the hotfix push in the evenings, Pacific Standard Time"
Re:Facebook is down
by kmicklas on Tuesday June 18, 2013 @09:43PM
(#5903265)
Facebook seems to have major reliability issues in general for me. At least once a day the site will not load for several minutes, despite my connection being fine. I also seem to be the very last person to get every new feature, e.g. graph search, hashtags, feelings on statuses, etc. (I don't yet have any of those...)
Source: Friend who interned at FB.
"Every week, usually on Sunday evenings, we stage our weekly push, which happens on Tuesday evenings, on the beta tier. This provides ourselves and developers 48 hours to test and find bugs before the build hits production. After Tuesday, while the beta tier is continuously updated with hotfix patches which are queued up for the daily hotfix release, the production tier is updated with the hotfix push in the evenings, Pacific Standard Time"
https://twitter.com/search?q=facebook%20down
Their "Sorry, something went wrong." page is copyright 2012.
01:18GMT - Seems to be loading every now and then.
01:25GMT - Loading consistently.
--
http://developers.facebook.com/live_status
Says facebook is healthy. You can see a spike in response time and drop in error rates(hehe) in the graphs at the bottom of the page.