News for hackers. Stuff that matters.
Re:Immutability, MVCC, and garbage collection by jasonwatkinspdx on Sunday December 29, 2013 @04:14AM (#6978966)
When talking about architecture at this level, you need to take a longer view. Once upon a time, a gigabyte database was mammoth beyond thought. Also, you're taking it as given that your data has its present storage footprint even though alternative architectures might have very different properties. It's worth noting that a purely additive database can be very aggressive about compression.

But also, petabyte datasets are downright cheap to deal with these days. They'll look more like that half gigabyte source code repo sooner than I think you're considering.

Re:Immutability, MVCC, and garbage collection by RyanZAG on Sunday December 29, 2013 @07:36AM (#6979387)
We're obviously coming from different backgrounds here - do you work at Google or similar ?

'Downright cheap' is a Heroku instance or two. 'Affordable' is 20 or so servers with a few TB of drive space each in a cluster with a few backups. Petabytes of data needs a huge number of servers and dedicated personal to look after - I don't know how that can be regarded as 'downright cheap'.

If you're in a position to handle petabytes of data you're probably going to roll your own data center with custom software ala Google's bigtable. This definitely isn't the market that these little startups are targeting as far as I can see. The OPs critique is very well founded.

Re:Immutability, MVCC, and garbage collection by corresation on Sunday December 29, 2013 @12:16PM (#6980257)
petabyte datasets are downright cheap to deal with these days

Are you from the future? In no universe is a petabyte database remotely close to being cheap, even for large organizations. At this point I am seriously questioning your industry knowledge.

Re:Immutability, MVCC, and garbage collection by empthought on Sunday December 29, 2013 @01:49PM (#6980684)
Sure it is, a few thousand entries pointing to 10-100GB pirated HD video blobs. What's the problem?

/s