Re:Billions wasted on Hadoop startups, the same will eventua...
by throwaway082729 on Friday May 31, 2019 @10:23AM
(#20061134)
I'm genuinely curious. Can PG handle hundreds of users querying 100s of GB, sometimes the same set of tables, at the same time?
Re:Billions wasted on Hadoop startups, the same will eventua...
by anarazel on Friday May 31, 2019 @10:36AM
(#20061266)
Well. You're going to run out of CPU and memory bandwidth pretty quickly. So you'd need replicas to share processing load.
But I honestly don't think hundreds of users each querying 100s of GBs is all that common.
Re:Billions wasted on Hadoop startups, the same will eventua...
by throwaway082729 on Friday May 31, 2019 @12:16PM
(#20062239)
Again, genuinely curious to learn if you've experience with enterprise companies using 'big data' technologies.
Aggregating 100s of GB isn't much of a problem for PG these days. Yes, you can be faster - obviously - but it works quite well. And the price for separate systems (duplicated infrastructure, duplicated data, out-of-sync systems, ...) is noticable as well.
But yea, for many petabytes of data you either have to go to an entirely different system, or use something like Citus.
Disclaimer: I work on PG, and I used to work for Citus. So I'm definitely biased.