Re:An open-source alternative to Mixpanel
by lindesvard on Tuesday May 21, 2024 @05:37PM
(#40434321)
Correct! Currently no batching but no need to make things complicated for now.
Can extend the current redis queue to batch request or clickhouse batch engine.
Looking forward to have these problems hehe.
Please DM if you want to chat about best approach!
Or enable async inserts on the ClickHouse client, which won't require changing to the buffer table engine.
https://clickhouse.com/docs/en/optimize/asynchronous-inserts
Re:An open-source alternative to Mixpanel
by lindesvard on Tuesday May 21, 2024 @08:28PM
(#40435992)
Have read about this, but totally forgot about it! Will try it out!
Re:An open-source alternative to Mixpanel
by klaussilveira on Tuesday May 21, 2024 @06:54PM
(#40435084)
Won't this be a problem as soon as a website reaches, say, 600 RPS? Which is not much.
Re:An open-source alternative to Mixpanel
by lindesvard on Tuesday May 21, 2024 @08:30PM
(#40436004)
I have time to grow. Don’t think I’ll hit this level just yet.
Re:An open-source alternative to Mixpanel
by jack_riminton on Wednesday May 22, 2024 @02:33AM
(#40437947)
This is the correct answer! any early project needs to make big trade-offs and premature optimisation has killed many projects. You'll never please everyone (especially on HN) but you're not making it for everyone
Re:An open-source alternative to Mixpanel
by FridgeSeal on Wednesday May 22, 2024 @05:36AM
(#40439089)
As someone using clickhouse, for similar event ingestion stuff - I’d lean towards implementing it sooner rather than later.
It’s not too hard, and it can surprising _how much more_ Clickhouse likes bulk queries.
Re:An open-source alternative to Mixpanel
by lindesvard on Wednesday May 22, 2024 @06:27AM
(#40439337)
Sure thing! Will look into it more during the coming weeks. Feels like I have the most of the dashboard complete now.
But I really want to start giving the native app some love as well
Dunno, that’s already a fairly big website. The one I was working on in the past with 15k customers did around 15RPS. The large enterprise system I work on with 3k users does slightly less than that. That covers an awfully large amount of use cases already, and then you can get 30x more?
So any major website will have problems with it. I would suggest moving this into a stream (NATS, Kafka) or just having some sort of service that buffers events, then offloads to Clickhouse in batch.