Maybe, but it's awkward. Graph or tree structures are still painful to store in relational databases, and a lot of problems turn out to involve those shapes.
> We have a working, declarative query language that Just Works(TM), for which we have written very good optimizers.
Maybe, but the tooling is still terrible. Where are the libraries? Where are the IDEs? Where's the integration that makes it easy to call procedures in my application language from SQL and vice versa? General-purpose programming languages have got better and better in the last 25 years, while SQL has stayed static.
> So, to sum it up: Exactly why should we abandon this?
Look at the ORM problem. Why do people continue to use these horrific bloated, leaky tools? Because it's really nice to be able to store and query in the native language of your application. We need storage tech that's better at supporting this.
A RDBMS is a very slow key value store, so anything that can't do transactions or can't do anything else, is always going to be faster at its extremely limited set of abilities.
From an engineering perspective if you need 10 HP to run your 5 KW generator, a RDBMS is like installing a 10000 HP marine diesel and then not having any idea how to start or maintain it or even where to get the fuel. Obviously every objective performance metric would be better for a 10 HP lawnmower engine in that app, if all you'll ever plan to use is 10 HP and you have no idea how to use anything more advanced anyway.
There are also very loudly trumpeted anecdotal situations or contrived thought experiments where certain unusual technologies fit a unusual situation very well. This is, oddly enough, unusual.
I've read and experimented with the "7 DBs in 7 weeks" book and it IS very interesting but I can't find any business cases to actually use any of it, which is somewhat frustrating. And my experience is how you end up with people writing CRUD apps to store cooking recipes that none the less use NEO4J because they really, really, want to add a line to their resume that they used NEO4J, not because the app needed it.
"I don't know what I'm doing, but someone who knows what they're doing anecdotally solved a completely different problem using tool XYZ, so for lack of any better idea, lets copy them".
If you're familiar with cargo cult science there is an enormous miasma of cargo cult engineering fogging up the entire database arena not just nosql.
Another concept that needs to be in the discussion is the "no silver bullet" rule from programming applies to database design, like it or not. Can't just sprinkle magic nosql pixie dust on any old random problem and expect it to work, any more than applying any random programing fad to any random problem will work.
The (old and new) tools are actually pretty interesting, although often poorly engineered (by the end user) and implemented. Its the persistent anti-patterns and non-engineering design technique that I'm properly arrogant and condescending toward.
Hammers are a cool new invention and have some great unusual new applications, but they don't install deck screws any better than the legacy screwdriver. Laborers on the job randomly mixing screws nails screwdrivers and hammers on the job and then internet discussions about how hammers and/or screwdrivers suck is nearly physically painful to watch.