Ah, the "not invented here" syndrome!

There are tons of things that you could do "in a couple of weeks" that more or less work. However, it doesn't mean you have to or even that it would be a good idea.

If all developers adopted the attitude that you have expressed, there would be thousands of sad sad developers who need to maintain shitty in-house analytics system because someone once said "I could do it in a week". There are tons of awful CMSes already because someone once said "I could do better than wordpress" / "I could create a better framework" / etc.

In a lot of the cases, GA is just good enough. Sure, you might need to spend some time to explore its features (custom dimensions, etc), there's more to GA than a number of pageviews for a given day. There are cases when GA is not enough. Fair enough. But it's definitely not the majority of the cases.

Sure, it makes sense for SpiderOak given it's target audience. However, there's no need to make such a generic statement about 'anyone working in the tech'.

Then the question is do you really want to maintain the infrastructure required to run the analytics smoothly? Especially if your company has dozens of millions of pageviews a month and depends on the real time needs (extra infrastructure to support that).

Are you familiar enough with the stack so you could have a high degree of confidence that you can fix productions issues which are inevitable? Quite often, an honest answer here is 'no'. Then can you afford to lose a few hours/days/weeks (whatever it would take to fix the issue) of data? Again, often the answer here is 'no'.

Of course, you have hosted solutions. But they are no better than GA in terms of privacy.

Paid support exists too but the cost can skyrocket pretty quickly, on top of paying for the infrastructure and maintaining it.

Not to mention you get perfectly accurate analytics, with no loss due to request blockers or disabled javascript.

The code for this is generic. An open source solution costs nothing beyond some CPU to process the logs and a database to store the analytics.

(1) https://support.google.com/analytics/answer/3125360?hl=en&re... (2) https://support.google.com/analytics/answer/2819948

a: https://tools.google.com/dlpage/gaoptout b:https://chrome.google.com/webstore/detail/do-not-track/ckdcp... b: I don't have the study off-hand, but IIRC some guy after finishing his masters from Stanford wanted to assess how much information Google had re: an average users browser history. The findings, based off Common Crawl data of the top 100k sites + presence of GA.js yielded something like ~> 75% of the web was tracked (not to be confused with how much of an end-user's traffic is tracked, that number will be far higher) based on sites with a GA.js history factoring in Referer tags. Those were unweighed numbers, i.e., I bet more than one out of two 45 year old woman's traffic can be analyzed to a 95% degree of completely entirely based off of Pinterest, Facebook, search history and the outbound links from her e-mail.

I've had "simple foss analytics" on my todo-list for quite some time. I'm hoping one can build on what piwik have collected wrt bot agent strings, ips etc - and combine with a simpler collector (adding php to the stack just for analytics isn't very appealing, never mind a php codebase of somewhat questionable quality).

Snowplow looks good, but I'm not sure if they have a supported "self-host" stack yet (they started out very awz/s3 centric).

I actually think there's room for a new product, that puts a little bit more thought into what questions it makes sense to ask, and how best to answer them (eg: does collecting metrics on every visitor even make sense if you can answer the same wuestions just as well by doing random sampling? You might want to quantify where your bandwith goes - but simple log analysis might do that easily enough - and it might have very little to do with your human visitors etc).

Perhaps systems administration is somehow very cheap for you, but I'm willing to bet it is still not "nothing" - even if the cost is you personally not watching a TV show you like because you're patching the web server on your analytics box for your personal vanity domain, that's still a cost.

For most operations, sysadmins are somewhat expensive, and because of that, busy. This is why Urchin was such a good idea, and why Google bought them - the proposition is to trade your users' privacy for the admin time it takes to support another internal app. There's an absolute no-brainer, assuming you don't care about your users' privacy (IIRC, they were going to sell the service before Google ate them, but that's ancient and trivial history).

If you're business is so small that an additional low-volume web server just to display your analytics (you don't need one for the actual tracking) is a big deal, then the same web server that serves your product can serve your analytics. Not a big deal.

I applaud SpiderOak, but they are much different from most other sites. They have privacy conscious customers to begin with, this is something that is good press for them and probably a net positive on their bottom line for doing it, not the case with most other sites. Also it's something they are doing after having a very mature product for many years, clearly not the first or most important thing they needed to tackle as a company.

GA is mostly used by people that don't need it, yet want to pretend they get actionable data out of it.

1. https://piwik.org/

I bring this up because people had been slamming moot for using GA on 4chan instead of piwik without understanding why.

Look at the comments from sandfox and afterlastangel in this thread. afterlastangel is pushing a billion, sandfox is around 300 MM per month.

http://forum.piwik.org/t/high-traffic-piwik-servers-database...

People seem to ignore that the tracking JavaScript is not what you're paying for. It's the backend + servers.

http://forum.piwik.org/t/high-traffic-piwik-servers-database...

Piwik is still using (unsalted) MD5 for passwords in 2015, and probably will still be using unsalted MD5 in 2016.

I can't believe unsalted MD5 is "by design" (https://github.com/piwik/piwik/issues/8753).

Considering Piwik is used by the GCHQ, I find it hilarious.

I don't find it using poorly implemented hashing in the administrative interface to be at all relevant to what they're doing, or why they shouldn't be using it.

Given these known security flaws, it's not a stretch to assume anyone who can see the GCHQ's Piwik server can have that data too, regardless of whether they are authorized.

See below for a small preview of what an attacker could exfiltrate (dissident IPs redacted for a reason):

https://firstlook.org/wp-uploads/sites/1/2014/02/piwik2.png

While we're talking about poor security practices: the privileged username in the screenshot is apparently still the default ("admin"), so I hope the password isn't still "changeMe" ... http://piwik.org/faq/how-to/faq_191/

Pretty much works like Google Analytics but utilises both client JavaScript and embedded runtime code to generate a richer picture of what is going on.

Too bad the interface on the Azure Portal is terrible. They spent too much time making it look fancy, and not enough time getting the 101s of usability right (which is a criticism I'd lay at the feed of the new Azure portal in general).

https://en.wikipedia.org/w/index.php?title=List_of_web_analy...

Probably the vendors of the software concerned. Perhaps it started out as a list of three with a major bias towards a particular product. And then the competitors responded, moderators did their things and eventually an accurate list was evolved.

Am looking to use this in lieu of Google Analytics.

EDIT: Sorry, I've been dealing with uBlock Matrix for too long, and forgot how advanced the other blockers pattern matching is. See the many responses to this for better information.

From EasyPrivacy[1]

    /piwik-$domain=~piwik.org
    /piwik.$script,domain=~piwik.org
    /piwik.php
    /piwik/js/*$domain=~piwik.org
    /piwik1.
    /piwik2.js
    /piwik_
    /piwikapi.js
    /piwikC_
    /piwikTracker.

[1] https://easylist-downloads.adblockplus.org/easyprivacy.txt

Every service does. Pingdom, GA, Olark, Github...

It took them a few weeks to write their own analytics. What features did they not implement? How many people worked on it?

Does your 1 or 2 person startup have 4 weeks to write their own analytics package or do you have more important stuff to do? (I'm betting you do. Like launching your product instead of re-inventing the wheel with analytics)

It's almost never "developers" who are deciding to use GA; it's middle managers or marketing departments.

So it's not hard to imagine marketing wanting it; presumably it provides them a lot of value that wouldn't be easy to recreate in-house.

That maybe no option...

GA is rather deep, with tons of integration and ways to slice and segment data.

Yeah, maybe in a few weeks you can get _something_ that'll give you something that'll make some manager not too unhappy. Seems like a terrible value prop for almost all companies since, unfortunately, approximately no one cares (or they run adblock anyways).

Not hard to track page hits, time on, time off, and arbitrary events.

EDIT:

Seriously? Folks, it's a table for analytics events, a few SQL queries to do basic reporting (at least in Postgres), a little bit of client-side JS to post the events, and a bit of server-side code to create the routes and maybe display the report page.

I guess if it doesn't include Kafka, Mesos, Kubermetes, Neo4j, and Docker, it isn't delivering business value.

ffs.

People are prematurely optimizing if they fear is "but but but mah datamoose".

Also, it's not prohibitively costly if you do even slight batching of the events, say batch load between every five minutes or an hour.

I'd love to hear somebody with war stories chime in though!

You're missing the point of why "rolling your own shitting [sic] implementation" is worth it: it's not the speed, it's the privacy.

Frankly most of what i read out of the tech world these days seems to be about pandering to developer laziness.

All manner of APIs and services seem to exist in their current form simply to extract rent from developers that don't want to do back end "dirty work".

The entire idea behind writing a Service as a Software Substitute[2] is about extracting rent.

[1] https://en.wikipedia.org/wiki/Rent-seeking

[2] http://www.gnu.org/philosophy/who-does-that-server-really-se...

Unless I'm mistaken, one big difference is that not using Google Analytics means you don't know which Google search pages people used to access your website. That can be a really important difference for some websites.

Sure, the basics are easy. But marketers and business people want to drill into a lot of data which is non-trivial to gather.

Unless you have a compelling business case (which SpiderOak does), it's not worth it.

I've recently been faced with this problem, and a solution doesn't have to be too complex.

There are roughly two parts to an analytics solution: event logging and, well, the actual analytics.

Writing your own logger in javascript is super simple, you're just sending off json objects to be inserted into a elasticsearch cluster. Since you have to define that logging anyhow, the only extra work you need to do is the layer to do the actual ajaxrequests.

What's left is running and defining your queries in elasticsearch.

BAM! Analytics

I realize it's not fit to be used for every situation, but it can so some pretty complex things this way without the hugest amount of effort ...

I don't think anyone was saying that GA is always better, it's just more often than not it is. It takes some skill and quite a bit of experience to draw the line at a reasonable place and correctly recognize the trade offs.

But then I started to fall short on disk space for storing too many events. This is a problem.

[1]: https://github.com/fiatjaf/microanalytics

[2]: https://github.com/fiatjaf/microanalytics-cli

    much of what we used analytics for anyway

So, I think, as more and more people will start using ad blockers, site owners will start getting less and less accurate stats from Google Analytics, forcing them to implement their own solutions. Hopefully, open source solutions will start providing the best features that Google does.

I have gone through their trial, but now I think I will register for the Solo account ($6/mo).