News for hackers. Stuff that matters.
Re:Using Analytics on My Website by the_gipsy on Friday December 15, 2023 @04:43AM (#38652562)
Just use GoAcces for fuck's sake. You don't need more than access logs. What are you going to do with that analytics data anyway? How much can you even trust that it's real and not bots that are getting better every day?

It's bad enought that marketing departments has convinced 99% of companies (or has overruled product / engineering) that it's good to track and make websites slower in the process.

Re:Using Analytics on My Website by sofixa on Friday December 15, 2023 @06:30AM (#38653095)
> Just use GoAcces for fuck's sake. You don't need more than access logs. What are you going to do with that analytics data anyway? How much can you even trust that it's real and not bots that are getting better every day?

I can provide you a very simple example. You have a blog, with articles. On a multi-thousand word article, you know that traffic that spent <30 seconds on it are either bots or people that didn't read through. If that's most of your traffic, your articles aren't very good or your introductions suck. If almost all mobile traffic is like that, your site probably looks bad on small screens.

You get the point, I hope. Access logs are useless for this kind of basic information.

Another very good example - there was a blog from someone involved in the UK GDS initiative, that described the case of a woman sitting with a handheld console in the waiting room of a government office. Initially you think she's just playing, but peeking over her shoulder, you see she's filling an application for unemployment. The browser on the handheld is horribly outdated, but she probably doesn't have another choice. Therefore knowing your audience, and adapting to it is crucial. From them:

> people now access GOV.UK in many different ways - 16,500 visits came from games consoles in the last month (Xboxes/PlayStations/Nintendos) - including 65 sessions from a handheld Nintendo 3DS (this is in 2015 https://gds.blog.gov.uk/2015/11/20/2-billion-and-counting/)

It entirely depends on what your website is doing if you need analytics or not. But a lot of orgs and people actually do, and can and should make use of them.

Re:Using Analytics on My Website by account42 on Friday December 15, 2023 @09:49AM (#38654708)
You don't need analytics to know that you should build a responsive website and use best practices to keep it resource usage small enough for old devices.
Re:Using Analytics on My Website by sofixa on Friday December 15, 2023 @10:37AM (#38655165)
> You don't need analytics to know that you should build a responsive website

You do need to know if you have users using their fridge's custom browser developed by some intern long since retired in 2012 that doesn't support HTML5 properly and needs weird hacks to actually be responsive.

> resource usage small enough for old devices

Which can prevent you from adding new features, or even just video in good quality which can choke an old device's browser. Graceful degradation is nice, and should be preferred whenever possible, but sometimes not possible. So knowing how many, and if any, of your users are using such things you have to go out of your way to support, is important.

Re:Using Analytics on My Website by johnmaguire on Friday December 15, 2023 @10:38AM (#38655176)
I don't think the Nintendo DS / handheld gaming device comments were about keeping resource usage small. Outdated browsers don't support all the same APIs that the ones today do. Typically, businesses will choose a cutoff that supports most traffic they receive from their users. Without analytics, you can't determine this cutoff.

It varies wildly from product to product.

Re:Using Analytics on My Website by joshyi on Friday December 15, 2023 @01:01PM (#38656729)
While JS analytics may suit the needs of marketing professionals concerned with product sales and article visibility, those focused on precise traffic patterns, like daily devops, find GoAccess to be a more effective tool.

JS analytics face an escalating risk of inaccuracies due to the myriad ways data can be manipulated, resulting in the generation of unreliable events from JavaScript endpoints. Keeping up with the industry reveals an emerging trend of blockers using AI-driven detection, diminishing the relevance of JavaScript analytics in such scenarios.

Re:Using Analytics on My Website by victorbjorklund on Friday December 15, 2023 @07:53AM (#38653645)
The benefit with using JS instead of access logs are that scraping bots, security scanners etc usually don't call your JS analytics so for a small site the number % of bots can be 90% of the logs.

And you can use your analytics data to better understand your users because again most bots won't trigger your analytics script (because why would they execute it and waste their CPU cycles on that?). And yes, you can't trust the exact numbers (because adblockers, etc) but you can see the trends:

Page A has 1000 visitors per day. Page B has 10 visitors per day. We can conclude that A is more popular than B (why that is another question).

Or Page A had 100 visitors on average last month and now it has 1000 visitors on average. We can conclude it got more popular. Etc.

Do you need that for your personal blog? Probably not. Do you need that on your e-commerce shop to verify wether you 1 million dollar ad spend makes a difference or not? Probably.

Re:Using Analytics on My Website by denton-scratch on Friday December 15, 2023 @09:49AM (#38654710)
> because why would they execute it and waste their CPU cycles on that?

The analytics script is not run using your CPU cycles; those belong to your victim/target.

Re:Using Analytics on My Website by joshyi on Friday December 15, 2023 @01:17PM (#38656886)
I think many developers overlook this aspect. If you're not in marketing, it's not just about tallying human visitors on our sites. It's essential to be vigilant against possible daily attacks and avoid excessive reliance on basic JS analytics.

Even for personal websites, relying solely on JS can compromise security. We require precise data, and logs prove more dependable than JS, especially with the increasing number of tools tampering with JS data.

We should steer clear of the trap of exclusively focusing on the count of human visitors.

Re:Using Analytics on My Website by libertine on Friday December 15, 2023 @06:40AM (#38653145)
> It's bad enought that marketing departments has convinced 99% of companies (or has overruled product / engineering) that it's good to track and make websites slower in the process.

Because there are no product tools and analytics to track and make websites slower in the process?

Re:Using Analytics on My Website by KronisLV on Friday December 15, 2023 @05:27AM (#38652780)
> Just use GoAcces for fuck's sake.

GoAccess seems pretty cool and is probably a good task for the job, when you need something simple, thanks for recommending it: https://goaccess.io/

Even if you have analytics of some sort already in place, I think it'd probably still be a nice idea to run GoAccess on your server, behind some additional auth, so you can check up on how the web servers are performing (in lieu of other options that aggregate your web server logs).

That said, I'd still say that the analytics solutions out there, especially self-hostable ones like Matomo, are quite nice and can have both UIs that are very easy to interact with for the average person (e.g. filtering data by date range, or by page/view that was interacted with), as well as have a plethora of different datasets: https://matomo.org/features/

I think it can be useful to have a look at what sorts of devices are mostly being used to interact with your site, what operating systems and browsers are in use, how people navigate through the site, where do they enter the site from and how they find it, what the front end performance is like, or even how your e-commerce site is doing, at a glance, in addition to seeing how this changes over time.

As for performance, I guess it depends on whether you care about any of the above, whether they actually help you make your site better. If performance was the top goal, we probably wouldn't be using Angular, React or anything like that in the first place, but only very lightweight options. Or, you know, not putting ads or auto playing videos on the sites either.

People have also said good things about Plausible Analytics as well: https://plausible.io/

Re:Using Analytics on My Website by bluish29 on Friday December 15, 2023 @07:45AM (#38653587)
Actually you don't even need Goaccess running all the time. They parse nginx logs directly so you can only run it when you want to do it (although in practice it takes sometime to parse and create graphs).
Re:Using Analytics on My Website by joshyi on Friday December 15, 2023 @12:55PM (#38656671)
I would contend the contrary. Our company consistently runs GoAccess because we are primarily concerned with unusual traffic patterns. This involves monitoring for potential attacks or abnormal bot traffic that might impact our servers, etc.

While JS analytics tools might be of more interest to the marketing team, they are not as crucial for the devops team.

Re:Using Analytics on My Website by joshyi on Friday December 15, 2023 @01:22PM (#38656945)
I believe JS analytics tools serve marketing well, providing estimates like who's purchasing my product or how many are reading my article.

However, for devops, relying too much on these tools can be a trap, potentially causing security and server issues without immediate detection. Our team opts for goaccess for this reason. We prioritize accurate data from logs and scrutinize traffic patterns, focusing beyond just distinguishing between human and bot counts, a task we leave to the marketing/product department.

Re:Using Analytics on My Website by closewith on Friday December 15, 2023 @06:03AM (#38652968)
If you use access logs for user analytics, you need all the same consents and adhere to the same ePrivacy/GDPR rules as JavaScript/Cookie-based tools. Once you build all the necessary tooling to use logs lawfully for user analytics, it won't be simpler than the commercial tools.

> It's bad enought that marketing departments has convinced 99% of companies (or has overruled product / engineering) that it's good to track and make websites slower in the process.

Marketing departments need the tracking to use the commercially available advertising, primarily from Google, Meta, and Bytedance. The tracking increases the return on advertising spend by 10-100x and for many businesses is required to be profitable.

This is the typical HN comment from an overconfident developer who believes everything they don't personally like must be the result of idiocy and incompetence.

Re:Using Analytics on My Website by stephenr on Friday December 15, 2023 @05:16AM (#38652720)
I love the concept of goaccess but last time I checked it still didn't have the ability in the web ui to present statistics from a subset of dates relative to the entire captured set.
Re:Using Analytics on My Website by AndrewStephens on Friday December 15, 2023 @10:17AM (#38654971)
GoAccess is nice but when I tried it a few years ago I found that the results were not accurate. Maybe on a large site the signal rises way above the noise but I found for my blog/project site (with maybe 20 hits a day) the counts consisted of obvious bots that GoAccess didn't filter correctly.

Google Search Console is also widely inaccurate in the other direction.

I ended up just implementing a simple hit counter[0] which was fine for what I wanted.

[0] https://sheep.horse/visitor_statistics.html

Re:Using Analytics on My Website by joshyi on Friday December 15, 2023 @12:51PM (#38656609)
If accuracy refers solely to human-vs-bot detection, there might be a point, but for comprehensive traffic analysis, access logs are unparalleled in accuracy.

JavaScript analytics are increasingly prone to inaccuracies due to numerous methods available for manipulating data, leading to the generation of inaccurate events from JavaScript endpoints. Staying current in the field reveals a growing trend of blockers employing AI-driven detection, rendering JavaScript analytics less relevant in these cases.