Because this data adds up and I later need to query / report on it, I'm taking it out of redis after it reaches a certain age (every night) and bulk inserting (archiving) it into postgresql with a bit of aggregation. This gives me the advantage of super fast writes using redis, but also super fast SQL queries for reports that don't have to be real time.
This is probably more hassle than you want to deal with for most situations, but when you do need those fast writes, and you're not super concerned if you lose some data if the server craps out, it seems to be a nice fit.
I'm very happy with the performance so far.
Redis will only evict data when it's explicitly removed or expired, and it will store as much data as you put into it.
memcached will evict the least-recently used data based upon memory pressure, and will only use as much memory as you configure it to.
They overlap in functionality, but I find they work better in complement to each other than having one replace the other. Redis for data you want to persist, possibly with a timeout -- user sessions, for example. memcached for data you want to cache, as long you have the free memory for it.
Probably not as fast as memcached, but one situation where it would be appropriate is if you're already using redis in your stack for something else (fast writes). Rather than adding another piece of complexity to your stack you can instead get double use out of redis as a cache as well.
maxmemory-policy allkeys-lru
How do I know? We ( http://bu.mp ) push 100s of GB though redis every day in LRU mode.
Basically, I exactly agree with the GP's use cases. ACID for things I want to keep, redis for small things that need to be fast (sessions, stats counters, etc).
Also, complex queries should be handled by performing them periodically on your backend stroage offline, storing the results in something like redis and then simply read from the redis DB online.
I think simple redis queries have the perfect level of data logic you would need just to be able to retrieve the corect/relevant results from the complicated queries you produce.
There are so many people who have written about methods on how to do something like this, eg. facebook, disqus, etc.
The author's mistake is that he is setting up a strawman argument. I've yet to hear Redis claiming to be a end all be all solution for data storage. In fact, it would make sense that most production environments use Redis as the non-persistent cache layer backed by an ACID compliant datastore for the time being.
The author's second point is that Redis fails "at scale in production environments". I'm curious to see some examples here of production Redis failing and I'd like to know how those companies were using Redis. Either way, the scalability debate should wait until 2.2 is released, because 2.2 is supposed to address primarily multi-server Redis instances.
I think the main problem is just believing that there can be "one true datastore". Different technologies excel at different things. You pick what you want your app to do, and then take a look at the tools given to you and pick the best one for the job.
Redis happens to be a great choice for many things, but ACID compliant storage is not really one of them at the moment, although I suppose you could set up 2 Redis instances, one for fast in-memory usage and one that syncs to disk on every command for now.
From my experience, Redis is great for the following:
- Session storage
- basic LRU cache
- Publish / subscribe
- key value store where you want to manipulate data in the form of common data structures
- simple / medium complex queries
Redis is not good for:
- ACID compliant data storage AND fast in-memory access at the same time (the holy grail)
- Complex sql-like queries
Right now, if you try to make Redis persistent in the event of a crash (fsync constantly), you will get a performance hit because the writes to disk will be too frequent.
Antirez is trying to solve the problem of persistence while still keeping fast in-memory access. The author does not feel like Antirez will succeed here, and became "angry", which I find is a bit unreasonable.
I personally use Redis in production for what it's currently good for. If the persistence with speed issue is solved, all the better. It's not an all or nothing type of situation. The worst thing the author could do in this situation is to pick SQLite as your one and only data store for the entire stack. There's nothing wrong with having multiple types of datastores. In fact it is the norm and trying to fight against it at this time is kind of pointless.