News for hackers. Stuff that matters.
Re:RAM Is the New Disk by ericfrenkiel on Monday July 31, 2017 @01:47AM (#14889734)
Yes, it's why we at MemSQL added a column store on disk in 2014. Memory-only is too limiting and has evolved to a notion of "memory-first."
Re:RAM Is the New Disk by AndyNemmity on Monday July 31, 2017 @03:03AM (#14890018)
Really a sort of odd choice though. I'd have gone the other way around with column store in memory, and row store on disk.

Not that I think that's ideal either though, having both in memory for hot used data, and the rest on disk is ideal. With an extremely easy to use setup that makes it essentially automatic, but with rules engines for finer grained tuning.

Re:RAM Is the New Disk by manigandham on Monday July 31, 2017 @03:09AM (#14890040)
MemSQL actually adds an in-memory rowstore to each columnstore for rapid ingest of new rows until they get compacted into a new segment. Columnstore data is pretty fast so it works well off disk compared to row stores which aren't as efficient.

SQL Server similarly has the hekaton in-memory tables + columnstore indexes and the latest version allows combining both for in-memory columnstores.

Re:RAM Is the New Disk by AndyNemmity on Monday July 31, 2017 @03:13AM (#14890059)
I've used MemSQL, and it's rapid ingest by the default isn't ACID compliant, so it sort of depends on how you compare it.

The results of the columnstore data was pretty fast, and it's even faster in memory. Depends on what you're doing, and what the requirements are.

Was really impressed by MemSQL, and loved the wire compatibility with mysql, so don't take this as just a knock on MemSQL in anyway.

Re:RAM Is the New Disk by manigandham on Monday July 31, 2017 @10:55AM (#14892549)
> by the default isn't ACID compliant

What do you mean?

Re:RAM Is the New Disk by AndyNemmity on Monday July 31, 2017 @05:15PM (#14895957)
I'm surprised you wouldn't understand this as it's an absolutely requirement given you are a user of MemSQL.

But then I went to their docs to link you to the details, and it feels like they intentionally avoid stating clearly the problem.

Essentially, they allow committed transactions to hit memory and not disk. They allow you to configure it so that's not the case, but it isn't the default, and looking over the current documentation they certainly aren't clear about it like an open source project would be.

transaction-buffer needs to be set to 0 for durability, but the way the docs are explaining it is trying to confuse not being durable, as a different kind of durability.

I'm not interested in getting into a long discussion about this though, but it's difficult to explain the literal issue when they do such a marketing job of trying to hide the specifics.

Now I'm far less surprised a user wouldn't know this. Apologies for my forward initial statement.

Re:RAM Is the New Disk by manigandham on Tuesday August 01, 2017 @12:07AM (#14897812)
It's not a surprise, that's normal. Durability just means writes are safe, how it does so doesn't matter. All databases use a write-ahead log with small batch/async flushes and this is the same with MemSQL.

The enterprise edition has HA so with data on 2 nodes for safety. Otherwise you lose the whole point of in-memory performance (for writes) if you're going to write every single bit to disk immediately.

They explain it clearly on the durability page: http://docs.memsql.com/docs/using-durability-and-recovery