News for hackers. Stuff that matters.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by markosaric on Tuesday October 06, 2020 @05:50AM (#24696149)
Hello HN!

We started developing Plausible early last year, launched our SaaS business and you can now self-host Plausible on your server too! The project is battle-tested running on more than 5,000 sites and we’ve counted 180 million page views in the last three months.

Plausible is a standard Elixir/Phoenix application backed by a PostgreSQL database for general data and a Clickhouse database for stats. On the frontend we use TailwindCSS for styling and React to make the dashboard interactive.

The script is lightweight at 0.7 KB. Cookies are not used and no personal data is collected. There’s no cross-site or cross-device tracking either.

We build everything in the open with a public roadmap so would love to hear your feedback and feature requests. Thank you!

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by baobabKoodaa on Tuesday October 06, 2020 @12:23PM (#24699471)
Hey markosaric! Very interesting product and I'm really tempted to try it out. One suggestion to add to your documentation: you should mention drawbacks in your "comparison" pages.

For example, the page where you compare Plausible to Google Analytics lists only advantages and doesn't indicate that Plausible could in any way be worse. But based on the description of how Plausible works, I'm fairly certain that people behind VPNs will not be properly counted (everyone with the same user-agent will be counted as if they were the same person). Use of VPNs is fairly widespread, so lack of fingerprinting will actually prevent you from accurately assessing how many visitors a site has.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by markosaric on Tuesday October 06, 2020 @12:46PM (#24699805)
Thanks!

Depends really on the perspective you look at it from. That post is from the visitor privacy / privacy regulations perspective. If you use GA full on, you will get some advantages such as storing cookies on devices but if you either make GA "privacy friendly" (enable IP anonymization, disable user-ID, disable cookies etc) or ask visitors to consent to being tracked you will get less accurate numbers than our method.

We haven't noticed any issues with VPN usage as still you need people to use the same IP, same user agent and visit the same domain all on the same day to be counted as one with us.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by jakemal on Tuesday October 06, 2020 @10:10AM (#24697795)
Love the product! Since you asked, one request that I would find helpful is having the stat comparisons from the previous day based on the same time the previous day rather than the total. The comparisons almost always show some -X% because you are comparing a 24 hour period yesterday to a 24 - X hour period today.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by markosaric on Tuesday October 06, 2020 @10:59AM (#24698333)
thanks! makes sense and i'd like to see that change too! i've added the request to our github now so we'll see what can be done: https://github.com/plausible/analytics/issues/344
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by bishalb on Tuesday October 06, 2020 @07:56AM (#24696760)
How do you track sessions without cookies? IP address?
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by markosaric on Tuesday October 06, 2020 @07:58AM (#24696768)
We generate a daily changing identifier using the visitor’s IP address and User Agent. To anonymize these datapoints, we run them through a hash function with a rotating salt.

hash(daily_salt + website_domain + ip_address + user_agent)

This generates a random string of letters and numbers that is used to calculate unique visitor numbers for the day. Old salts are deleted to avoid the possibility of linking visitor information from one day to the next.

See full details here: https://plausible.io/data-policy

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by JimDabell on Tuesday October 06, 2020 @09:37AM (#24697507)
Doesn’t this mean that if you accidentally serve your website on two domains (e.g. example.com and example.net with no redirect) it will count one visitor twice if they visit both domains? What is the benefit to including the domain in the hash?

Not really keen on the use of the IP address. I’ve been behind load balancing proxies and weird mobile networks often enough to know that I can appear from a dozen different IPs in the space of an hour just by browsing the web normally with default settings.

Have you considered requesting a 24 hour private cacheable resource and counting the requests on the server? Or is the browser cache too unreliable?

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by ukutaht on Tuesday October 06, 2020 @10:06AM (#24697757)
More accurate would be that we use the site_id in the hash. You can serve your website from many different domains but as long as the site_id in your script tag is the same, the hash remains the same.

The site id is included in the hash to prevent cross-site tracking. Otherwise the hash would act almost like a third-party cookie and people could be tracked across different sites.

> Not really keen on the use of the IP address.

Yeah, it's not ideal. We do check the X-Forwarded-For header so as long as the proxies are being good citizens, the client IP is present in that header.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by dustinmoris on Tuesday October 06, 2020 @11:57AM (#24699108)
> Have you considered requesting a 24 hour private cacheable resource and counting the requests on the server? Or is the browser cache too unreliable?

Nice idea.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by benbro on Tuesday October 06, 2020 @08:23AM (#24696943)
Is it possible to track users over a week or month? Often the same user return to our website several times before buying and it's important to learn about this behavior.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by markosaric on Tuesday October 06, 2020 @08:27AM (#24696975)
no. that's the decision we made in order to be as privacy focused as possible. there's no way to know whether the same person comes back to a site the day after or later. they will always be counted as a new visitor after the first day.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by dustinmoris on Tuesday October 06, 2020 @12:01PM (#24699157)
Not sure if privacy is a valid argument here. As long as visitors are annonymised and it is self hosted I don't see how this would invade a person's privacy.

The problem with Google Analytics is not that they track a user on a single domain, the problem is that users are tracked on a *.google.com domain and Google knows who you are based on your other google sessions and knows everything you do on the internet because every website uses GA. With a self hosted product that wouldn't be the case, so the privacy is given by nature even if you'd track a user across a space of a month or longer.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by XCSme on Tuesday October 06, 2020 @12:28PM (#24699553)
I agree, self-hosting and decentralizing analytics data is the biggest improvement to user privacy that can be done today and has multiple other benefits for both the user and the webmaster.

Now, if only the EU would push more against data decentralization instead of writing cookie policies that result in horrendous user experience on the web...

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by lmkg on Tuesday October 06, 2020 @02:50PM (#24701063)
That is not how Google Analytics works. Its cookie (the Client ID) is a first-party cookie set on the domain of the website hosting it.

Google Analytics optionally integrates with both Google Ads and DoubleClick, and both of those integrations do a cookie-match against .google.com or .doubleclick.net cookies. But those integrations are optional and off by default.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by propogandist on Tuesday October 06, 2020 @08:47AM (#24697112)
this will limit your solution from many deployments.

Knowing returning visitors from first time visitors is quite important and helps to asssess if viewership, audience and customer base is growing over time.

For startups the "how many unique visitors do you get in a month" may be an important KPI and you're saying your solution cannot answer this question, so another solution will be needed to be deployed.

Unique visitor data's also needed to assess effectiveness of campaigns and run e-commerce operations. There's often campaigns to bring back a user who previously didn't buy (email, ads etc). It's important to measure the effectiveness of these investments separately in web analytics given the campaigns will be different for new and recurring visitors.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by gabriel34 on Tuesday October 06, 2020 @09:13AM (#24697310)
Correlation will hold between unique hashes and unique visitors to access increase in unique visitors for your campaign. Even the percentage of these accesses that are returning is likely constant. All you would be giving up is the ability to measure variation in the returning percentage across several days (even so you could probably modify the code to change the salt every x days without losing much of the privacy benefits)

There will always be value to be extracted from the invasion of your users' privacy, but you also hit diminishing returns over this increasingly invasive probing. Plausible is aiming for "good enough" whilst respecting people's privacy, and that is a good compromise IMHO.

There is a trade-off. You will never get 100% of the information without all the tracking, but there is information that represents more bang for the privacy buck.

Would you not have a acceptable error increase in your decisions with a bit less information and a lot less privacy invasion?

EDIT> I think more control to the user is better, so instead of canvas fingerprinting, shady cross-site tracking and all, I would rather have a uuid that my browser informed, but that I controlled, so I could be anonymous when I want to and be tracked when I don't care, or when I genuinely agrees it adds value.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by open-source-ux on Tuesday October 06, 2020 @10:51AM (#24698236)
"this will limit your solution from many deployments."

I think tech companies (particulalry Google) have conditioned us to expect analytics [1] to be an essential component of all apps and web services. Developers have happily accepted this, rather than questioned it (unless they happen to be the ones being tracked). But actually, analytics may not need to be as detailed (or as intrusive) as many think it needs to be.

Here is a blog post from Whimsical (an online flow chart tool) who decided to remove Google Analytics:

> "We realized that all our tracking stuff had barely marginal value. We were just accumulating data because it might be useful someday. And because everybody else was doing it. Yet 100% of our product decisions over the past year were based on strategy, qualitative feedback and our own needs." (My emphasis)

From: https://whimsical.com/blog/choosing-privacy

[1] Words like 'Analytics', 'Telemetry', 'Web Beacon' etc are examples of the dishonesty of the tech industry in using words to hide their real purpose and soften their impact. All of these words are about tracking online behaviour, but no-one would dare use the clearer, more honest word – tracking – in their app or web copy.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by ponderingfish on Tuesday October 13, 2020 @07:24AM (#24764424)
Apples and oranges. You have taken the example of a SaaS tool who might not need that level of information. But, a newspaper or a magazine absolutely needs deep information in order to sell advertising and sponsorship on their site.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by markosaric on Tuesday October 06, 2020 @09:11AM (#24697295)
yeah, i understand. we're trying to have a balance between these:

1. privacy of site visitors

2. compliance with privacy regulations

3. useful and actionable data for site owners

it's difficult to track people from visit to visit or from one device to another without breaking the first two (cookies, browser fingerprinting...) so we had to make some decisions.

in general sites that try to get visitor consent to cookies and/or to tracking realise that majority of them don't give it, so even the data that may not be as accurate as full on tracking becomes very valuable.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by thirdsun on Friday October 09, 2020 @05:37AM (#24728256)
The problem is that I most likely have to get consent (opt-in) from the user in order to collect that data - at that point I can either trick my visitors into giving it by using various dark pattterns or I end up with a dataset that is pretty much useless since nobody will opt-in.

If those are my options I prefer not having to annoy my visitors with consent banners over extended and persistent analytics data. That's why we actually migrated from Google Analytics to Plausible.

By the way: Import of analytics data from Google would be a great feature if anyone from Plausible is still reading here.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by jononor on Tuesday October 06, 2020 @11:34AM (#24698815)
If you have a customer relationship, then they can sign in (and accept your ToS) - and then you can, if they allow, track them that way. Similarly for re-activation campaigns, one can use something like UTM. Don't need to track everyone on the internet with generic stats mechanisms (like Plausible provides) for that.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by dbbk on Tuesday October 06, 2020 @10:02AM (#24697718)
It's an intentional limitation. If you want that data, go with Google Analytics or some other option.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by rkwz on Tuesday October 06, 2020 @09:36AM (#24697495)
> hash(daily_salt + website_domain + ip_address + user_agent)

That's a pretty interesting technique!

I built a privacy friendly analytics project https://github.com/sheshbabu/freshlytics and I've been wondering how to correctly count unique visitors to a website. I don't store cookies or any PII data at all so by definition, it's hard to distinguish between two different visits - are they from same person or different people?

An alternative approach is used by Simple Analytics - https://docs.simpleanalytics.com/uniques where they use referrer header to derive unique visits. They mention that they don't use IP addresses as they're considered fingerprinting.

But it looks like a hash function (whose salt gets rotated daily) strikes a good balance between fingerprinting while maintaining user privacy. Any downsides to this approach?

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by markosaric on Tuesday October 06, 2020 @11:07AM (#24698426)
downsides are basically that in order to gain the benefits for the user privacy and compliance with regulations, you lose a bit of accuracy depending on the situation. we cannot see whether the same person returns to a site on a different day so count them as a new unique visitor.

i assume that using the referrer header to count uniques has even more downsides as i imagine the number of unique visitors with that method would be much higher than it actually is.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by youngtaff on Tuesday October 06, 2020 @10:27AM (#24697970)
Doesn't that depend on visitors not sharing an IP address i.e. not behind something that does NAT?
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by markosaric on Tuesday October 06, 2020 @11:02AM (#24698372)
yes, that's a trade-off in this privacy first approach. if several people are on the same ip address, visiting the same website and having the same user agent on the same day, they look the same to us.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by Hitton on Tuesday October 06, 2020 @08:53AM (#24697153)
That seems pretty robust, but I have question about the salt. How does the daily salt part work? How is it stored during the day? Are historic salt values stored?
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by ukutaht on Tuesday October 06, 2020 @09:15AM (#24697328)
The salt is stored in a Postgres database and we run a daily job that generates a new salt and deletes old ones.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by vimes656 on Tuesday October 06, 2020 @01:57PM (#24700565)
It's so great the whole code is available under the MIT license.

After seeing many open source projects with paid services going for more protective licenses, like the AGPL or the newer eventually open licenses (i.e. Business Source License), I wonder what was your rationale to still pick the MIT.

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by markosaric on Tuesday October 06, 2020 @02:26PM (#24700823)
thanks! we're big fans of open source so wanted as permissive licence as possible.

one of the things we spend a lot of time thinking about is how to make our project sustainable while still being as open as possible in everything we do.

a bit more details in this post "How to pay your rent with your open source project" https://plausible.io/blog/open-source-funding

Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by tommoor on Tuesday October 06, 2020 @03:00PM (#24701160)
It's all fun and games until someone spins up a competitive hosted service using your MIT licensed codebase.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by ponderingfish on Tuesday October 13, 2020 @07:24AM (#24764434)
And ..... they've switched to AGPL.
Re:Show HN: Plausible – Self-Hosted Google Analytics alterna... by malisper on Tuesday October 06, 2020 @02:54PM (#24701112)
Q: Since users of Plausible are sending you their visitors IP addresses, doesn't that mean GDPR applies? GDPR mentions "disclosure by transmission" as an example of processing[0].

Since that's the case, doesn't that mean you need a legal basis for processing the data[1], you need a way to allow users to opt out of collection of their data[2], and your clients need to sign data protection agreements[3] with Plausible?

I tried looking for Plausible's point of view on this and how they address the above, but I couldn't find anything.

[0] https://gdpr-info.eu/art-4-gdpr/

[1] https://gdpr-info.eu/art-6-gdpr/

[2] https://gdpr-info.eu/art-21-gdpr/

[3] https://gdpr-info.eu/art-28-gdpr/