Relational databases solved the problem they had, of requiring much developer effort to evolve for new application needs, and re-organization of the datastore for changing performance needs. Basically, everything is a table, and you can combine/separate tables, from whatever form it currently is, to whatever form you currently need. This is also helpful for reporting (as opposed to operational applications).
But relational databases, like dynamic languages, pay a performance price for their flexibility.
Of course, every other innovation has since been added, and there's Oracle's licensing too. So it's much more complex than the base technology of relational algebra.
Honestly, IMO the biggest issue holding noSQL databases back is lack of good documentation/support a lot of the time. Technically, though, they're going to become the standard for most use cases soon.
There are hundreds maybe thousands of NoSQL databases ranging from Redis to MongoDB to Cassandra to InfluxDB to Druid. They pretty much have nothing in common other than storing data. Many of them support SQL directly, others via Spark and almost all have a SQL equivalent.
This doesn't require something relational, just something centralized. And not even that; consider Spanner.
Whereas implementing a Key-Value Store is orders of magnitude simpler. There is plenty of room for innovation, of course - but you don’t need 1,000 people to build-on some new replication scheme or distributed backend.
If you take a simple NoSQL system then tack on things like object scheme support, JSON blob value indexes, node graph support, etc you’ll end up with an RDBMS analogue.
So the barrier-to-entry to building a NoSQL system is much lower than a RDBMS - and for many companies they might decide to build their own rather than extend an existing system - leading to more completion but not necessarily a better product.
CockroachDB is the newest from-scratch SQL engine that I am aware of: https://www.cockroachlabs.com/
They re-use the RocksDB K/V store maintained by Facebook (in C++) as the storage layer, and their own code is written in Go, so I suspect that their development work is probably a lot less time-expensive than the teams working on older SQL databases.
Based on license revenue it's actually Oracle and MS SQL server by miles. They are currently #1 and #3 on DB-Engines.com. (https://db-engines.com/en/ranking) MySQL is #2. PostgreSQL is #4, though it's pretty far back from the first three.
I mean PostgreSQL is open source, and required no license fee. I'm not sure that license revenue is a good comparison metric.
It's popular on HN to focus on OSS products, but there's a very large proprietary RDBMS market measured both in terms of revenue as well as users. That's how Oracle and MS got to be the behemoths they are today.
So you basically want full consistency (vs eventual consistency) in a distributed environment. You can set up you noSQL that way. I don't see how relational DB's are "better" than noSQL in consistency.
The main reason technically-capable companies like Amazon or Netflix are moving away Oracle has nothing to do with horizontal scaling. It's the licensing costs. Oracle is just too expensive for them, and even if you are not Amazon yourself, RDS (or any other Cloud RDBMS) would be much cheaper for you.
I'm not sure this is true, you can look at the new features in 8i, 9i, 10g, 11g, 12c and see that Oracle is adding a lot of features, and some of them are very impressive. The problem is most people don't need most of them and if you want any of them you have to pay for all of them.
And the "non-traditional" databases had a pretty low bar. MongoDB just had to stay up for more than 5 minutes and not lose your data. Do they even do that yet?
What are you implying? Do you have any source for that? Some of the top tech companies out there use non-traditional databases, including MongoDB. They won't if the nodes crash every 5 minutes and/or there is data loss.
There are other non-traditional databases that are much better than MongoDB.
Amazon had a big outage in recent times that their own post-mortem linked to trying to move off Oracle to Aurora.
Interestingly it was posted here to HN and repeatedly and immediately flagged off the page - it appears that some HN readers are very keen on suppressing anything that paints Oracle in a good light. It makes you wonder what other stories might be going missing that would alter people's impressions of this company.
Andy Pavlo, database professor at CMU, has debunked it and gave details on the journalist's dishonesty.
Also, anything is better than oracle. If not for technical reasons, price wise it makes a lot of sense.
Your last sentence appears to be typical of the problem I'm describing.
I've seen such outages after an Oracle version upgrade so it's all but major.
For your information I've been managing critical databases for more than a decade so I know that it might look unusual to you, but it's quite clear to anyone barely knowledgeable about database management that the article really overblown the issue. It was also mocked by Amazon's CTO.
So please, manage databases for a few years. Then I might hear your opinion and maybe consider it.
To whit, de-normalised tables for the sake of "efficiency".
It's pretty difficult to under normalise a two column table -- which is what K/V is.
Yes, like traditional car companies not taking EV's too seriously, or Intel not taking mobile/ARM too seriously, etc. It seems the older and more established a company is, the thicker their forward-thinking blinders are. "We're king, no one can EVER touch us."
Now I see things differently -- the non-traditional databases are just better at scaling horizontally, eventual consistency, and running in cloud environments than the traditional databases. They are easier to set up and use. Some of them now have pretty good relational features and schemas.
During the past decade the Oracles of the world have continued to think dismissively of new non-traditional databases, as I did at first. The non-traditional databases got better and better while Oracle kept doing a lot of the same stuff. Oracle just didn't take the competition seriously, and that's fair enough because the competition didn't deserve to be taken seriously at first -- but now it does, and it's becoming a threat to Oracle's business.
It's obvious that Amazon would want to use their own database products -- but the impressive thing is that those products, which are not very old, are already good enough to replace Oracle for a lot of use cases at very high scale.