News for hackers. Stuff that matters.
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by mike_ivanov on Friday May 03, 2019 @11:53AM (#19819495)
The point of normalization is logical consistency, not IO optimization.
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by jchrisa on Friday May 03, 2019 @11:57AM (#19819535)
Sure. But the point of (early) NoSQL was IO optimization, not logical consistency.
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by acdha on Friday May 03, 2019 @01:36PM (#19820441)
I never saw it portrayed that way: it was always higher-level programmer ease of use pitches along the lines of “isn’t this easier than doing a JOIN?” pitch shortly before the speaker had a valuable learning experience about the value of data integrity and consistency.
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by virgilp on Friday May 03, 2019 @02:06PM (#19820752)
You really don't remember the "web scale" memes about Mongo? It was always primarily about "performance".
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by acdha on Friday May 03, 2019 @02:22PM (#19820913)
I don't remember seeing that predicated as a low-level I/O savings — it was always things about being easy to cluster or avoiding the performance overheads of ACID. The pitch was usually “it's easier and you don't need a skilled DBA”.
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by adrianhel on Friday May 03, 2019 @08:02PM (#19823325)
What I heard was "You can practically scale indefinitely". Which I guess is more true of NoSQL. As is I/O savings. But it's not something I've ever heard it advertised for.
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by rhizome on Friday May 03, 2019 @04:40PM (#19822066)
Performance was an imposed callsign of Mongo, the reason everybody started using it was the declarative schema stuff and so the dev didn't have to know how to start a terminal, which was a not-insignificant hurdle. It doesn't have significant performance differences over other simple KVs. "Faster" was primarily a measurement of learning curve.
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by bogomipz on Friday May 03, 2019 @12:17PM (#19819688)
I was curious about the parenthetical "early" in your comment. Is this optimization no longer the case then? Could you elaborate on recent developments regarding this? I've been out of the NoSQL loop for some time and am genuinely curious.
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by tracker1 on Friday May 03, 2019 @12:29PM (#19819800)
The problem is, it varies... you have what are effectively key/value stores to document databases, column stores and everything in between. You have systems built on other systems. RethinkDB and Cockroach have different approaches than Redis, Mongo, Cassandra or others.

CockroachDB gives an SQL interface over the top of a distributed data store with better consistency and relations. Cassandra has no real relations over a BigTable/Column-Store solution. It really just depends.

I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments.

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by dorlaor on Friday May 03, 2019 @01:46PM (#19820553)
We at Scylla tune the kernel, drives, stride, clock source automatically and benchmark the disk to figure out what's its maximum parallelism so it won't queue (we can saturate 8 NVMe) and we'll do the queuing within the DB's userspace so we can prioritize real time queries over background tasks:

https://www.scylladb.com/2018/04/19/scylla-i-o-scheduler-3/

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by bogomipz on Friday May 03, 2019 @07:31PM (#19823181)
>"I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments."

I think this is already happening. I think it was Aerospike that used the FTL of of NANd/NVMe drives for a direct key value store and I think another vendor maybe Fusion had an SDK for this as well. The Optane stuff looks really interesting, are there any server vendors shipping with those?

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by Rafuino on Monday May 06, 2019 @03:06PM (#19842654)
Not OP but, yes, server vendors are shipping with Optane DC Persistent Memory, like Lenovo [1] and Supermicro [2], among others. Google also has a testing program [3, 4] with access to instances with 7TB of total memory.

[1] https://lenovopress.com/lp1066-intel-optane-dc-persistent-me... [2] https://www.storagereview.com/supermicro_superserver_with_in... [3] https://cloud.google.com/blog/topics/partners/available-firs... [4] https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVi...

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by jchanimal on Friday May 03, 2019 @12:35PM (#19819877)
By early NoSQL, I mean the projects that are basically an API wrapped around a data structure, that push the understanding of the data structure onto the developer. Newer products (like my employer FaunaDB) prioritize correctness and offer abstract query interfaces like GraphQL.
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by bunderbunder on Friday May 03, 2019 @12:49PM (#19820028)
This one gets hard to pick into, given how nebulous the term NoSQL is.

One could probably argue, quite convincingly, that the point of truly early NoSQL was that SQL hadn't been invented yet.

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by sieabahlpark on Friday May 03, 2019 @11:58AM (#19819539)
It's both.
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by mike_ivanov on Friday May 03, 2019 @12:18PM (#19819702)
No, it is not. Normalization is an artefact of Relational Algebra (a set theory being applied to tabular data). It has nothing to do with concrete implementations and their concerns (such as IO). http://wiki.c2.com/?RelationalModel.

Moreover, ALL existing relational databases do denormalize data behind the scenes (think about caches and indexes) for the sake of IO optimization, which means that normalization does not achieve the IO optimization goal by itself (otherwise, why bother?)

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by bunderbunder on Friday May 03, 2019 @01:02PM (#19820126)
It really is.

Originally, schema normalization was just about logical consistency. But that was before query optimizers were invented, and long before they grew to be so very optimized for working with normalized schemata.

Nowadays, even someone who cares 100% about performance and 0% about data integrity (And is somehow still using a relational database, yeah. So what? It's MY hypothetical.) still has good reason to default to something like 3NF. Insert famous Knuth quote here.

Caches and indexes and all that stuff may technically count as denormalizations, but, insofar as the definition of 3NF doesn't mention them, that's a tangential issue.

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by crazygringo on Friday May 03, 2019 @03:41PM (#19821588)
> Nowadays, even someone who cares 100% about performance and 0% about data integrity (And is somehow still using a relational database, yeah. So what? It's MY hypothetical.) still has good reason to default to something like 3NF

That's just... wrong.

Best practices with database design are to start with a totally normalized schema because it reduces scope for errors/inconsistencies.

And then to intentionally denormalize where required to increase performance, in order to reduce lookups from joins or computing aggregate functions -- at the cost of having to manually ensure duplicated data remains consistent. Query optimizers can only do so much.

(If it weren't to increase performance, I can't really think of any reason why you ever would denormalize on a single database server, unless you have some extremely complicated calculations that SQL isn't capable of expressing?)

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by bunderbunder on Friday May 03, 2019 @04:11PM (#19821831)
> And then to intentionally denormalize where required to increase performance

Hopefully under profiler guidance, like for any other optimization. For my part, the frequency that I find that trying a denormalization helps performance has been on a downward trend for a while now. Mostly down to just straight-up duplicating data at this point. Databases really are getting very good at what they do.

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by ncmncm on Friday May 03, 2019 @11:36PM (#19824281)
This makes more sense than it seems to.

MongoDB's insight was that the overwhelming majority of data stores don't matter enough to justify worry about correctness. It doesn't go faster than other DBs if you turn on the "do it right" flags, but their customers mostly don't.

Would you be willing to wait an extra five seconds for page load to be sure the banner at the bottom of "people who looked at this also looked at these" list is fully up to date and correct? Or is filling the boxes with any old crap good enough?

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by acdha on Saturday May 04, 2019 @07:11AM (#19825749)
> Would you be willing to wait an extra five seconds for page load to be sure the banner at the bottom of "people who looked at this also looked at these" list is fully up to date and correct?

This is a separate question — do you do a live query or pre-compute it? — and it’s not really significantly easier to do that with Mongo’s model.

The pitch I saw was usually based on it being easier than having to think about your data model in advance, which is relevant to your example: everyone I know who picked it did something like that, thought it was less work and that was great, and then had some problem which came down to copies of data getting out of sync and so e.g. the “also looked at” box had the wrong price or a typo which had been fixed elsewhere. Fixing that usually cancelled out what claimed performance advantages and then some.