News for hackers. Stuff that matters.

A comment on No to SQL? Anti-database movement gains steam . Parent comment

Re:No to SQL? Anti-database movement gains steam by gaius on Thursday July 02, 2009 @06:45AM (#684142)
It happens with large RDBMS systems anyways after about 5GB of data, same type of scalability decisions have to be made.

More like 5Tb these days.

[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by drawkbox on Friday July 03, 2009 @03:42AM (#685755)
I should have clarified, I meant around 5GB per one type of data. Maybe a table or dataset that needs to be merged. For instance if you had a product database table that needed to be joined with another GB large or set of data at runtime.

At that point, even in RDBMS you have to stop with the joins and flatten. I have experienced these limits in Oracle and MSSQL around 20-30 million row tables.

Scalable fundamentals like database flattening, dimensional modeling etc are all what key value stores give you from the start. But really a good mix works best or project specific, just saying in the future with TB, PB of data the JOIN is a historic remnant.

It is possible one day that RDBMS will be seen as one of those evil optimizations that we made in our small relative worlds at the time.

[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by gaius on Friday July 03, 2009 @10:13AM (#686045)
I've tables much, much larger than that.

It is possible one day that RDBMS will be seen as one of those evil optimizations that we made in our small relative worlds at the time.

I doubt that for the simple reason that relational databases are maths: the relational algebra and relational calculus. There's no such theoretical underpinning to object databases.

[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by drawkbox on Saturday July 04, 2009 @05:35AM (#687051)
I doubt that for the simple reason that relational databases are maths: the relational algebra and relational calculus. There's no such theoretical underpinning to object databases.

True, the relational architecture is moved to the object or code level, if you think about it in relation to size, it makes more sense there for the future.

The RDBMS was the brain but it is also the storage, code will now dictate how to use the storage and not the storage itself, the storage is becoming a component.

[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by sho on Thursday July 02, 2009 @07:20AM (#684176)
Well, the disk space is not the controlling parameter usually. Getting a ridiculously huge amount of fast, reliable disk is pretty much a solved problem. If all you were doing was writing logs at a constant rate then you could "scale" pretty much indefinitely just adding larger and larger disks to the same poky DB server, and it wouldn't matter if it was 5GB or 5PB.

The proper scaling issue is, of course, with the cpu and memory of your single DB server - and its disk speed - which can only be expanded so much, and at exponentially increasing cost. It's conceivable that even with a fixed amount of data, say 100G, you might have to confront scaling issues with increasing site usage, long before you reached anywhere near the limits of your raw disk space.

[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by gaius on Thursday July 02, 2009 @09:20AM (#684316)
One database I work on was under 2Tb when I got it a little over 3 years ago and is nearly 30Tb now, so I am very familiar with the scaling issues. No need for sharding or any of that nonsense either!
[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by trezor on Thursday July 02, 2009 @11:37AM (#684559)
That sharding is required once your DB exceeds what you have of physical RAM seems to be a irrevocable meme in the MySQL world.

Seeing the amount of people generalizing that to apply to all RDBMSs makes me think they don't have any DB-knowledge outside MySQL.

[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by wvenable on Thursday July 02, 2009 @01:23PM (#684753)
That's crazy advice even for MySQL. Having a DB that exceeds physical RAM is why you use a DBMS.
[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by sho on Thursday July 02, 2009 @03:44PM (#685028)
Who said that? Certainly not me, and I doubt gaius has 30TB of RAM so not him either. How much RAM you need is a complex question and revolves to a certain extent around the DB caching the "hot points" but differs so radically app by app there's really no rules or general guidance you can point to. It comes down to measurement of your particular installation, as always.

And fyi I prefer PostgreSQL. But MySQL isn't THAT bad, c'mon.

[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by trezor on Thursday July 02, 2009 @07:02PM (#685349)
I'm not sayin MySQL is that bad, but I exclusively hear this kind of advice and anti-DB rhetoric from people running MySQL and MySQL only, who are complaining about scaling.

MySQL may be able to perform better, as I haven't done in depth tests I wouldn't know how far it can be stretched, but my point is that almost everyone complaining about the performance of the RDMS model seems to come from MySQL thinking that it represent the utmost limits of what relational databases can do, and that is quite sad.

[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by sho on Friday July 03, 2009 @03:54AM (#685764)
Hm. Well, I agree with your observations. But maybe they're more willing to suggest alternatives because they haven't invested so much into learning an alternate system? You would expect someone who has, say, spent the last 15 years becoming an expert on DB2 to refrain - even subconciously - from questioning the very fundamentals his career is built on. MySQL users may not be advanced but at least they are unbiased.

MySQL is not that bad, no. With proper knowledge it can do a lot, and beyond its limits, capably configured, I am not sure any RDBMS can help. Facebook is not developing Cassandra because they can't afford Oracle.

[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by gaius on Friday July 03, 2009 @05:06AM (#685816)
I first used MySQL a decade ago... There are plenty of people who've fully invested their careers in it. Which is as sad as someone who thinks MS Access is the state of the art.
[ Parent | on HN ]
Re:No to SQL? Anti-database movement gains steam by sho on Friday July 03, 2009 @05:23AM (#685833)
Sigh. Can't argue with that.
[ Parent | on HN ]