News for hackers. Stuff that matters.

A comment on New in PostgreSQL 10. Parent comment

Re:New in PostgreSQL 10 by imtringued on Thursday September 21, 2017 @04:50PM (#15307229)
To be honest 6 TB of data isn't that much. I'd say big data starts at around one petabyte because one petabyte requires 3 servers with 45 [1] 8TB hdds. At this point your data is so large it's impossible to store it on a single physical machine.

"big data" usually doesn't require transactions though or at least no transactions across servers so you could probably get away with sharding postgresql via citus.

[1] http://www.45drives.com/products/storage/

[ Parent | on HN ]
Re:New in PostgreSQL 10 by SEJeff on Thursday September 21, 2017 @06:21PM (#15307850)
I agree with this response. Those aren't very big either though. The Dell DSS7000 series servers[1] support 90 x 8TB drives (We run a ceph cluster on 4 of the DSS7500 storage nodes).

[1] http://www.storagereview.com/dell_announces_availability_of_...

[ Parent | on HN ]
Re:New in PostgreSQL 10 by Thaxll on Thursday September 21, 2017 @08:41PM (#15308527)
Except that it's just stupid to do that, you don't want to store that much data in one host, it goes against any sane architecture design, good luck when you're mb / cpu dies. Splitting data on smaller / cheap hosts is the way to go for ops / scaling / backups.
[ Parent | on HN ]