News for hackers. Stuff that matters.
Re:Announcing Apache Spark 1.4 by ericfrenkiel on Thursday June 11, 2015 @04:10PM (#9702031)
Check out MemSQL's Community Edition for this very use case. We shipped MemSQL with an open-source multi-threaded, bi-directional connector to Spark.

The DB has two storage engines: in-memory row tables, and on-disk column tables for efficient compression and permanent retention. Then, it becomes an easy task of INSERT/SELECT...FROM to move data from memory to disk very quickly.

There are no comments on this comment yet.