Instead of using a map (or COMPACT STORAGE), they should have defined their schema upfront (one of the limitations brought on by not-SQL). However if they didn't want to do that then COMPACT STORAGE is obviously a better solution than using a Map.
To answer your question about COMPACT STORAGE, standard CQL basically does the work for you (in terms of parsing the "xsv") but you have you define the schema upfront (as in, you have know all the map keys before hand). The reason they tell you not to use COMPACT STORAGE is for those cases where you don't need a dynamic schema, using COMPACT STORAGE doesn't really get you anything.
Lastly, IMO, I wouldn't touch CQL Collections or COMPACT STORAGE unless absolutely necessary. If you do need a dynamic schema I would rather encode the data as a msgpack or protobuf blob.
We tried. It was not only less disk efficient, but also slower for queries. Wasn't the result we expected, but alas.
However, I'm even more surprised at the "slower for queries" part. Maybe I'll do some tests with COMPACT STORAGE.
Would have gladly gone into more detail, but at 4,000+ words for the blog post already... :)
I stand by my point that the mistake was using CQL maps when you should have just used a defined schema.
My concern with Redis is reliability. Redis Cluster has problems with consistency [1], whereas unclustered Redis -- well, it syncs to disk every 10 seconds or so, but even then I'm concerned that the reliability of its on-disk structures haven't been as battle-tested as, say, PostgreSQL. Or has it?
Yeah, about that:
http://redis.io/topics/persistence
> Using AOF Redis is much more durable: you can have different fsync policies: no fsync at all, fsync every second, fsync at every query. With the default policy of fsync every second write performances are still great (fsync is performed using a background thread and the main thread will try hard to perform writes when no fsync is in progress.) but you can only lose one second worth of writes.
If you are happy with a Master/Slave setup and occasionally having to deal with data loss due to Master failures [e.g. losing a second or two of data that wasn't replicated], it works nicely.
Just realize I wouldn't use Redis as a long term persisting of data because of issues like this:
Redis is neet if your information fits in RAM, but that's not what Cassandra is for.
You can easily get commodity servers with 256GB of RAM per node x 10 shards. Cluster or not, as per your use case.
row key = key
column = field
I know Lua is "an option" in Redis but I avoid it like the plague because under production loads, Redis hashes perform and are easy to maintain w/o Lua.