News for hackers. Stuff that matters.

A comment on I/O Is Faster Than CPU – Let’s Partition Resources and Eliminate OS Abstractions [pdf]. Parent comment

Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by tracker1 on Friday May 03, 2019 @12:29PM (#19819800)
The problem is, it varies... you have what are effectively key/value stores to document databases, column stores and everything in between. You have systems built on other systems. RethinkDB and Cockroach have different approaches than Redis, Mongo, Cassandra or others.

CockroachDB gives an SQL interface over the top of a distributed data store with better consistency and relations. Cassandra has no real relations over a BigTable/Column-Store solution. It really just depends.

I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments.

[ Parent | on HN ]
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by dorlaor on Friday May 03, 2019 @01:46PM (#19820553)
We at Scylla tune the kernel, drives, stride, clock source automatically and benchmark the disk to figure out what's its maximum parallelism so it won't queue (we can saturate 8 NVMe) and we'll do the queuing within the DB's userspace so we can prioritize real time queries over background tasks:

https://www.scylladb.com/2018/04/19/scylla-i-o-scheduler-3/

[ Parent | on HN ]
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by bogomipz on Friday May 03, 2019 @07:31PM (#19823181)
>"I think with what's coming out of SSD/NVMe and even Optain DIMMS, that there will be databases directly tuned to control/set their own data storage in these environments."

I think this is already happening. I think it was Aerospike that used the FTL of of NANd/NVMe drives for a direct key value store and I think another vendor maybe Fusion had an SDK for this as well. The Optane stuff looks really interesting, are there any server vendors shipping with those?

[ Parent | on HN ]
Re:I/O Is Faster Than CPU – Let’s Partition Resources and El... by Rafuino on Monday May 06, 2019 @03:06PM (#19842654)
Not OP but, yes, server vendors are shipping with Optane DC Persistent Memory, like Lenovo [1] and Supermicro [2], among others. Google also has a testing program [3, 4] with access to instances with 7TB of total memory.

[1] https://lenovopress.com/lp1066-intel-optane-dc-persistent-me... [2] https://www.storagereview.com/supermicro_superserver_with_in... [3] https://cloud.google.com/blog/topics/partners/available-firs... [4] https://docs.google.com/forms/d/e/1FAIpQLSeX1tN6Qt-aQUK2iVVi...

[ Parent | on HN ]