News for hackers. Stuff that matters.
Re:Surviving a traffic surge: Three techniques to scale your... by bmelton on Tuesday March 29, 2011 @09:04AM (#2382418)
Those tips are obviously worthwhile, but doesn't address massive scale for heavily dynamic sites. These should definitely be at the beginning of any optimization checklist for sure.

One thing I found interesting was his remark this:

Google Analytics failed to detect the surge: page load time was so high that visitors were closing the page before analytics could load.

He then remarks that it took Analytics 15 hours to detect the spike, but isn't that true of all Analytics instances? I'm not sure if the author is mistaking Google Analytics' delayed reporting as a fault or if I'm missing something.

Re:Surviving a traffic surge: Three techniques to scale your... by JoachimSchipper on Tuesday March 29, 2011 @09:19AM (#2382455)
These tips are for people getting slashdotted, not people building the next Google. Your average blog simply isn't "heavily dynamic", and if you're building Google/Facebook/... there are better resources.

Google Analytics is Javascript-based and thus vulnerable to people closing the page before it loads. It's actually near-realtime, but just doesn't work if you're having this problem. I'd imagine watching free Apache instances/threads, outgoing bandwidth or probably even the Apache logs would be more useful.

Re:Surviving a traffic surge: Three techniques to scale your... by jedsmith on Tuesday March 29, 2011 @10:36AM (#2382758)
> Google Analytics is Javascript-based and thus vulnerable to people closing the page before it loads.

And people with Javascript disabled, which is a startling number. The disparity between reported views in Analytics and what I can observe from my Web server logs is amazingly large.

Re:Surviving a traffic surge: Three techniques to scale your... by JoachimSchipper on Tuesday March 29, 2011 @10:45AM (#2382796)
Are you sure those aren't bots? Analytics packages vary widely in their ability to filter out bots.
Re:Surviving a traffic surge: Three techniques to scale your... by jedsmith on Tuesday March 29, 2011 @10:49AM (#2382812)
Bots have Javascript disabled too. :)

You're right, of course. I considered it and started grepping them out, but got bored and did something else interesting. From a casual glance, there were a lot more Mozilla UAs than reported views, though.

Re:Surviving a traffic surge: Three techniques to scale your... by rsloan on Tuesday March 29, 2011 @09:28AM (#2382480)
While the reason Google Analytics failed at this has already been explained, GA Ecommerce tracking is near real-time, but not error corrected. Data in other parts can take several hours to show up, Google says to wait 24 hours for data to be complete and error corrected.

Omniture SiteCatalyst is near-real time. I don't know if they do some error correction later like GA does for Ecommerce data.

Re:Surviving a traffic surge: Three techniques to scale your... by joelhaasnoot on Tuesday March 29, 2011 @09:30AM (#2382488)
GA data is delayed by a day, but if you adjust the date range manually, you can see the data that has come in so far. Not guaranteed to be complete, but some data.
Re:Surviving a traffic surge: Three techniques to scale your... by pdenya on Tuesday March 29, 2011 @10:37AM (#2382764)
Google Analytics updates hourly, reporting gives you only full days by default but you can see data much more up to date than that.