> Matomo is decent, but my main issue with it is the performance when run at any sort of scale. It's PHP/MySQL, which is nice for ease of self-hosting, but it means a lot of things need to be pre-calculated.

I've never actually run into performance issues, neither when using it in production professionally, nor for my self-hosted sites (with Matomo always running on-prem). I'd say the performance of PHP and MySQL/MariaDB is most likely decent as long as you don't go too far into specialized workloads, for example log aggregation/tracing; though even some APM solutions like Apache Skywalking also support using traditional RDBMSes for this purpose as well: https://skywalking.apache.org/docs/main/v9.0.0/en/setup/back...

That said, I can't help but to wonder at what actual scale (number of logged events/second, given certain hardware) you'd run into issues. Luckily, because adding basic analytics is usually quite easy, testing this for your own workloads shouldn't be out of the question - then you can let the data speak for itself.

The performance issues aren't with the measurement requests but with reporting.

When I eval'd it for my book last fall there were big delays in reporting waiting for segments and then also issues with custom reports. I think they have changed the default behavior to get around some of the former, but with MySQL it's always going to be tough for larger queries.

(if there's any performance issue on the measurement side it has more to do with the JavaScript payload because they include a lot in their standard JS bundle).

I don't think PHP/MySQL is the issue. I run a site with a single 3TB MySQL database and have no problem pulling and charting the data in fractions of a second. Some of the tables are in the area of 10 billion rows.

For comparison, I've also worked with a large variety of dashboards from companies using Snowflake/Tableau that are paying teams millions of dollars a year to build their interfaces, and it's normal to see sluggish load times that are taking 3-10s+.

The big difference is I know what charts and data I want to display, and I build my database from the ground up so it's optimized to show that data. These companies have teams of people that know absolutely nothing about databases and data structures, they just drag a bunch of data together to create some temporary table, then pass that data over to another piece of software. That gets imported into something else where someone that can barely write SQL writes a poorly optimized statement that somehow shows the expected result. They have more flexibility, and everything loads, although performance isn't ideal. Mine either works in 0.1s or it'll never load.

> with MySQL it's always going to be tough for larger queries

MySQL was too slow for running analytics on a single website? How much data are we talking about?

For self-hosted, if you are doing into the tens of thousands of pageviews per day you may want to turn off real-time and switch to auto-archiving which pregens reports. That level of traffic depends a lot though on the report period you are running against and how big you've provisioned your MySQL server. YMMV and I haven't benchmarked at multiple traffic levels.

What kind of scale are you looking at?

I'm building UXWizz, which is on the same stack, and in my experience on a single server you can track up to 10-15 million sessions total (~1M sessions per month, with 1 year retention). To scale, for now I made it so you can have one database per domain, so the data is sharded by domain (if you are tracking multiple domains).

The biggest bottleneck is querying a large dataset, the storing of data seems to work pretty well. You could of course archive the data and automatically create stats using cron jobs, but this leads to a lot less filtering options and not so fresh data.

The scaling issues I'm talking about are on the reporting side, not the measurement side. With Clickhouse you can do complex analytics queries on live data, with MySQL as you say you may need to pre-gen with cronjobs, etc.

Makes sense. My question was what magnitude of data amount are we talking about? Because for the average website, receiving less than 100k monthly session, a single MySQL instance is more than enough for real-time queries in sub-100ms query times.