News for hackers. Stuff that matters.
Re:How NoSQL forced the evolution of a scalable relational d... by threeseed on Friday July 13, 2018 @03:57AM (#17521218)
Not sure what you are talking about here.

There have been scalable and strongly consistent databases since the invention of the concept of NoSQL i.e. HBase and Cassandra (CL=ALL).

And the idea that "learning and optimize SQL" would instantly change people's rationale for using these databases shows you don't understand them much at all. There are many factors that come into play. For example you can't use SQL databases for large scale feature engineering.

Re:How NoSQL forced the evolution of a scalable relational d... by CuriousSkeptic on Friday July 13, 2018 @05:44AM (#17521624)
To me “large scale feature engineering” is precisely the kind of problem SQL excel at.

Is it something that makesme grab for SQL it is the need to do adhoc querying exploring data.

Care to elaborate?

Re:How NoSQL forced the evolution of a scalable relational d... by logophobia on Friday July 13, 2018 @07:38AM (#17522024)
He's probably refering to a combination of features that "nosql" databases have.

* Schema-on-read: Makes it easier to ingest large amounts of data, and then do ad-hoc exploration. The schema is only determined when reading the data, which is a bit easier for one-off data exploration, you determine how to interpret the data when actually using it. Not appropriate for production systems though. For example, a customer gives you a few TBs of data, you dump it on hadoop, and query it with spark. It would slow you down if you first have to convert it to a relational schema. Again, only good for one-off stuff.

* Most SQL databases have column limits, so if you have a very large amount of features, I'd imagine you'd run into these limits.

* Scalability. Feature engineering is very parallelizable, most normal SQL databases (excluding stuff like cassandra) aren't trivial to scale.

Re:How NoSQL forced the evolution of a scalable relational d... by logophobia on Friday July 13, 2018 @04:09AM (#17521265)
Cassandra is AP (available and partition tolerant), with tunable consistency. HBase is CP (consistent and partition tolerant, but not always available in case of network partitions).

Spanner is consistent, available and partition tolerant due to the extremely precise clock synchronization that google achieved on their hardware and network, an innovation I hope will spread. So yes, that's new.

I don't think I claimed sql was the only solution that ever made sense. I'm sure for some use cases, schema-on-read or no schema is appropriate. But most of those usecases are rather niche. NoSQL became popular for (web)application software, which in my opinion isn't an appropriate use case for most of these applications.

Edit: I misunderstood. Spanner is CP, but google claims here: https://storage.googleapis.com/pub-tools-public-publication-... that it's also practically CA, which is technically incorrect, and something I misunderstood before.

Re:How NoSQL forced the evolution of a scalable relational d... by saurik on Friday July 13, 2018 @05:33AM (#17521575)
> Spanner is consistent, available and partition tolerant due to the extremely precise clock synchronization...

This is nonsensical; even in a universe without relativity and with instantaneous communication--an unrealistic world where being able to talk about synchronized clocks can even make sense (as otherwise you fundamentally must deal with an inability to decide the order of two events due to latency)--you still cause indeterminate simultaneity by introducing a partition: your perfectly accurate and infinitely precise timestamps on one side of the partition would no longer be comparable with timestamps on the other side of the partition for purposes of establishing causality between events, as the recent changes being made on one side of the partition "should have" taken place between any older changes on the other side and any transactions being performed right now, and yet they couldn't because they were unknown. Spammer either gives up availability (waiting to verify what happened) or consistency (accepting that it will have to fix the result later) when a partition occurs: they can't do both, fundamentally; the only extent to which there is an engineering challenge here is "maybe you can minimize the number of partitions you have", which I totally believe Google to have done; but, given a partition (which cannot be fully prevented), the tradeoff is clear and unavoidable.

Re:How NoSQL forced the evolution of a scalable relational d... by qaq on Friday July 13, 2018 @05:29AM (#17521564)
Spanner is CP I think the A part comes from reliability of google network infrastructure which is not part of DB system's design per se.
Re:How NoSQL forced the evolution of a scalable relational d... by logophobia on Friday July 13, 2018 @05:39AM (#17521608)
I think I misunderstood that yes.

https://storage.googleapis.com/pub-tools-public-publication-....

TLDR: It's technically CP, but google claims partitions are so rare people can assume it's CA as well (99.999% available).

I assumed TrueTime bypassed the CAP theorem, but apparently that's marketing bs. It's to ensure something called external consistency, which is important if you want to take consistent snapshots over a distributed system.

Re:How NoSQL forced the evolution of a scalable relational d... by majewsky on Friday July 13, 2018 @09:00AM (#17522536)
Semi-related soapbox: Can we all please stop saying "CAP theorem" and instead say "CAP rule"? The formulation of this supposed theorem is so vague that it can neither be proven nor disproven.
Re:How NoSQL forced the evolution of a scalable relational d... by logophobia on Friday July 13, 2018 @11:22AM (#17523674)
It was proven in 2002, has a formal definition, so I'm going with calling it a theorem. Maybe this'll help: https://mwhittaker.github.io/blog/an_illustrated_proof_of_th... ?