News for hackers. Stuff that matters.

A comment on Ask HN: What database should I use for analytics data?.

Re:Ask HN: What database should I use for analytics data? by sixhobbits on Sunday November 03, 2019 @03:58AM (#21432853)
> This is all working fine

Are there any specific reasons that you think Postgres won't be able to scale?

I don't have personal experience solving the problem you're describing but if you do choose to use something other than Postgres I think Apache Kafka might be interesting for some parts of your infrastructure at least.

Segment [0] has written quite a lot about their tech in a similar space.

[0] https://segment.com/blog/the-10m-engineering-problem/

[ Parent | on HN ]
Re:Ask HN: What database should I use for analytics data? by hodgesrm on Sunday November 03, 2019 @11:52AM (#21434974)
Cloudflare documented their PostgreSQL to ClickHouse journey here: https://blog.cloudflare.com/http-analytics-for-6m-requests-p...

It covers a lot of the scaling bottlenecks in their original PG-based pipeline and how they eliminated them.

(Disclaimer: I work on ClickHouse.)

[ Parent | on HN ]
Re:Ask HN: What database should I use for analytics data? by AdriaanvRossum on Sunday November 03, 2019 @04:45AM (#21432988)
I think you are right, I'm kind of looking for a technical challenge where I actually should just improve the current setup. Thanks for the link!
[ Parent | on HN ]