Ask HN: What analytics do you use when Google Analytics reaches its limits?

Posted by dumbfounder on Saturday May 18, 2013 @03:01PM
from the ask-hn dept.

From Google's page on data limits: "Data collection limit: If you exceed 10 million hits per month, there is no assurance that the excess hits will be processed." - We are doing more than a million hits per day, so what now?

Right now Twicsy continues to use GA, and also Quantcast, but their numbers (on page views in particular) have started to diverge. On some days the page view counts are 60-70% different. We are not sure who to trust.

We have tried hosting Piwik ourselves and also a hosting provider, it just stopped working after a few days for some reason in both cases. Clicky doesn't support websites that get more than 1 million page views per day on one website. We can't find anything else appealing.

We don't have a lot of money to spend, at most we would like to spend a few hundred dollars per month. Is there something out there that is worthwhile at that price? If not, what are our alternatives?

( 13 comments | 9 points | on HN )

Re:Ask HN: What analytics do you use when Google Analytics r... by omarrr on Saturday May 18, 2013 @03:48PM (#5730138)

My suggestion would be to look into a workaround inside of GA.

Eg: Create a new suite within GA (UA-XXXXX-2), and use the new ID for a subset of your page views. Look into how your page views are happening across your site and plan on distributing the analytics to fall under 10M. For instance, use your main ID for landing, top and trending pages and a second one for the rest of the site. If you are near or over 20M views you might want to consider a third or forth ID.

Good luck!

===

Edit: On a second thought, one of our sites reports over 10M a month and we never run into issues. GA doesn't process more than a small percentage of visits for the reporting, so your issue might not be related to the 10M cap.

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by dumbfounder on Monday May 20, 2013 @10:20AM (#5737810)

Thanks for the suggestion, but I see a few issues with this approach: 1. You can't track the number of unique visitors anymore 2. I will lose the data history I have now with GA. I am actually not sure that it is incorrect, I just am suspect at this point and need validation. If it is indeed correct then I don't want a break in the data.

But, something to think about, so thanks.

Re: Edit: Are you sure? I mean, really sure?

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by centdev on Sunday May 19, 2013 @12:27AM (#5731773)

Are you sure it's GA? Our numbers differ between GA and quantcast because google isn't tracking opera mini on feature phones but quantcast is. We are running 450m pageviews a month through ga and its typically a day behind processing. Outside of that its missing feature phones, but I don't believe it's because of our numbers.

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by dumbfounder on Monday May 20, 2013 @10:24AM (#5737825)

Nope, I am not sure, so thanks for the input. The numbers are 60-70% different for us between Quantcast and GA, so I don't think opera mini would make up anywhere near that difference.

And, are you sure it isn't dropping data? I mean, really sure?

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by t0 on Saturday May 18, 2013 @03:08PM (#5729967)

Reddit uses GA. There may be some special arrangements you need to make. http://blog.reddit.com/2011/02/reddit-billions-served.html

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by itsprofitbaron on Saturday May 18, 2013 @06:03PM (#5730635)

They'll use the GA Enterprise version which serves up to 1 Billion "hits processed per month" its £90,000[1]/$150,000[2] as a flat fee per year.

[1] http://www.google.co.uk/intl/en_uk/analytics/premium/feature...

[2] http://www.google.co.uk/intl/en_us/analytics/premium/feature...

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by dangrossman on Saturday May 18, 2013 @03:28PM (#5730043)

Server access log analyzers (awstats, webalizer, etc)? Those are free, and get you most of the same basic reports. I don't know of any SaaS analytics that aren't out of your price range at 30M+ data points per month.

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by staunch on Saturday May 18, 2013 @05:47PM (#5730601)

http://clicky.com/

http://clicky.com/help/pricing#/compare/

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by dumbfounder on Monday May 20, 2013 @10:21AM (#5737817)

They only allow up to 1 million page views per day per site. We are over that number.

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by staunch on Monday May 20, 2013 @05:35PM (#5740206)

On the register page it says

"If your site has more than 1,000,000 daily page views, please contact us before registering!"

I take that to mean they'll work something out with you, and based on their pricing it should be well within your price range.

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by lifeguard on Saturday May 18, 2013 @06:18PM (#5730669)

Consolidated logs + Splunk.

http://www.splunk.com/

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by dumbfounder on Monday May 20, 2013 @10:26AM (#5737835)

Maybe I am wrong, but I don't think that analyzing logs is anywhere near as effective as the javascript based solutions that use cookies to track users. We receive a lot of bot traffic and I don't think log analyzers do a good job of telling the difference.

[ Parent | on HN ]

Re:Ask HN: What analytics do you use when Google Analytics r... by lifeguard on Tuesday May 21, 2013 @02:31PM (#5745739)

It is trivial to filter out the bots IMO. You can also correlate system logs with application logs.

[ Parent | on HN ]