News for hackers. Stuff that matters.
Re:Redis 3.0.0 is out by timdorr on Wednesday April 01, 2015 @10:52AM (#9304054)
From the release notes: https://raw.githubusercontent.com/antirez/redis/3.0/00-RELEA...

    * Redis Cluster: a distributed implementation of a subset of Redis.
    * New "embedded string" object encoding resulting in less cache
      misses. Big speed gain under certain work loads.
    * AOF child -> parent final data transmission to minimize latency due
      to "last write" during AOF rewrites.
    * Much improved LRU approximation algorithm for keys eviction.
    * WAIT command to block waiting for a write to be transmitted to
      the specified number of slaves.
    * MIGRATE connection caching. Much faster keys migraitons.
    * MIGARTE new options COPY and REPLACE.
    * CLIENT PAUSE command: stop processing client requests for a
      specified amount of time.
    * BITCOUNT performance improvements.
    * CONFIG SET accepts memory values in different units (for example
      you can use "CONFIG SET maxmemory 1gb").
    * Redis log format slightly changed reporting in each line the role of the
      instance (master/slave) or if it's a saving child log.
    * INCR performance improvements.
Re:Redis 3.0.0 is out by vasquque on Wednesday April 01, 2015 @11:19AM (#9304288)
You fogot:

WARNING: Redis 3.0 is currently a BETA not suitable for production environments.

Re:Redis 3.0.0 is out by james33 on Wednesday April 01, 2015 @11:24AM (#9304325)
I think he just forgot to remove that from the changelog as this is indeed marked as a stable release now, except possibly for the cluster feature.
Re:Redis 3.0.0 is out by antirez on Wednesday April 01, 2015 @11:38AM (#9304415)
Sorry, stale warning removed.
Re:Redis 3.0.0 is out by JonnieCache on Wednesday April 01, 2015 @11:12AM (#9304240)
Here's the full cluster spec: http://redis.io/topics/cluster-spec

Looking forward to seeing it taken apart by aphyr.

Congrats to Salvatore and the rest of the redis committers!

Re:Redis 3.0.0 is out by sylvinus on Wednesday April 01, 2015 @10:11PM (#9308277)
Then you might want to see both of them on the same stage in June :) http://www.dotscale.io/
Re:Redis 3.0.0 is out by legedemon on Wednesday April 01, 2015 @11:52PM (#9308580)
You made my day! I'm booking a slot right now.
Re:Redis 3.0.0 is out by poooogles on Wednesday April 01, 2015 @05:28PM (#9307045)
Is he still doing Jepsen tests? Thought he was doing Clojure work atm?
Re:Redis 3.0.0 is out by steveklabnik on Wednesday April 01, 2015 @06:22PM (#9307376)
Yes, he is not doing Jepsen at the moment. The end of his last blog post on it has some explanation. That said, I wouldn't wonder if he'd make an exception, given that it's Redis...
Re:Redis 3.0.0 is out by necubi on Wednesday April 01, 2015 @08:11PM (#9307889)
He's been tweeting about Jepsen tests on Aerospike lately: https://twitter.com/aphyr/status/582653874894282752.
Re:Redis 3.0.0 is out by chupy on Thursday April 02, 2015 @10:46AM (#9310976)
We are considering Aerospike because apparently a lot of our competitors are using it and I really would love to see those tests.
Re:Redis 3.0.0 is out by JonnieCache on Wednesday April 01, 2015 @06:52PM (#9307543)
Pretty sure I saw him tweeting about jepsen the other day
Re:Redis 3.0.0 is out by gsnedders on Wednesday April 01, 2015 @11:49AM (#9304502)
(To note, I have limited experience with using Redis. My questions may be stupid.)

As far as I can tell, most of the advantages of Redis come from the fact that it's all held in memory and so access is fast. Is networked access to other parts of the cluster quick enough that it is quicker than storing the data on one computer, partly on disk? When would one want to use a Redis cluster rather than something stored on-disk and cached in memory?

Re:Redis 3.0.0 is out by famousactress on Wednesday April 01, 2015 @12:05PM (#9304630)
The performance section of the spec[1] does a pretty good job explaining how the implementation remains fast.

    In Redis Cluster nodes don't proxy commands to the right
    node in charge for a given key, but instead they redirect
    clients to the right nodes serving a given portion of the
    key space.

    Eventually clients obtain an up to date representation 
    of the cluster and which node serves which subset of 
    keys, so during normal operations clients directly 
    contact the right nodes in order to send a given command.

    Because of the use of asynchronous replication, nodes 
    does not wait for other nodes acknowledgment of writes 
    (if not explicitly requested using the WAIT command).
[1] http://redis.io/topics/cluster-spec#performance
Re:Redis 3.0.0 is out by meritt on Wednesday April 01, 2015 @12:18PM (#9304718)
The primary advantage of Redis is native data types and access patterns beyond simple key-values: lists, sets, sorted sets, hash tables, pub/sub, hyperloglog, and scripting (lua) support.

Being memory-based was simply a feature but not necessarily something that set it apart: Memcached had that area pretty well locked down for being a blazing fast key-value store. And then Membase was basically memcached + persistence and clustering. Now Redis has clustering too!

Re:Redis 3.0.0 is out by siculars on Wednesday April 01, 2015 @06:33PM (#9307441)
seconded. memory = fast is not the point. the point is that it gives you specific data types that do specific things and since it's in memory it does those things very quickly. that said, you need to implement your own clustering or sharing solution. jury is still out on the new cluster code, i havent reviewed it yet.
Re:Redis 3.0.0 is out by mmahemoff on Wednesday April 01, 2015 @12:32PM (#9304820)
As well as the optimisations mentioned by famousactress, note that it's common to run Redis (or other in-memory databases such as memcached) on a separate server anyway. So there's already some network overhead but it tends to be small enough if the content is small and everything's co-located.
Re:Redis 3.0.0 is out by famousactress on Wednesday April 01, 2015 @02:06PM (#9305567)
Right, but I assume the commenter was more curious about the presumed multiple-network-accesses usually involved with a cluster which proxies to some other node to satisfy requests would do to the overall performance of the system... which is a totally reasonable question/concern.
Re:Redis 3.0.0 is out by agonzalezro on Wednesday April 01, 2015 @10:55AM (#9304087)
For lazy readers: CLUSTERING!
Re:Redis 3.0.0 is out by xofer on Wednesday April 01, 2015 @11:38AM (#9304418)

   Maybe it works great from day one,
   maybe it will need a few more iterations, 
   and possibly with 3.2 we'll improve support for many stuff, 
   but my guess is that Redis 3.0.0 today, in some way, changes what Redis is.
Re:Redis 3.0.0 is out by agonzalezro on Wednesday April 01, 2015 @11:48AM (#9304491)
It's software. Everybody knows how it works :)
Re:Redis 3.0.0 is out by andrewstuart2 on Wednesday April 01, 2015 @12:10PM (#9304664)
Noooo, apologies for the downvote. That "up" button is tiny and apparently easy to miss.
Re:Redis 3.0.0 is out by yankoff on Wednesday April 01, 2015 @11:22AM (#9304312)
I just hope it's not an April's fool joke
Re:Redis 3.0.0 is out by agonzalezro on Wednesday April 01, 2015 @11:55AM (#9304548)
The author wrote this on twitter: https://twitter.com/antirez/status/583279481453936640

"p.s. no April fools here. 1 year ago I released HyperLogLog, today 3.0.0, just to contrast with some shit done this April fool lameness."

Re:Redis 3.0.0 is out by ihsw on Wednesday April 01, 2015 @11:33AM (#9304385)
Coincidentally, the HyperLogLog redis type was introduced on April Fools last year.

http://antirez.com/news/75

Re:Redis 3.0.0 is out by beat on Wednesday April 01, 2015 @11:52AM (#9304519)
Yeah, clustered Redis, announced today? Sounds like a joke to me!
Re:Redis 3.0.0 is out by antirez on Wednesday April 01, 2015 @12:42PM (#9304903)
Not a joke, the release was due in these days, so I picked 1 April since we have a tradition now to ship 1 April. Last year with HyperLogLog support, because of the futuristic name of the data structure, people had an hard time to believe it was a really thing and not a joke.
Re:Redis 3.0.0 is out by alanh on Wednesday April 01, 2015 @01:05PM (#9305094)
I understand your reasons and nevertheless hope that you can realize that you are still contributing to April Fool's insanity by making it that much harder for people to tell what is serious and what isn't on this day of disregard.

You’re totally within your rights here. I just beg you to consider the audience.

Re:Redis 3.0.0 is out by antirez on Wednesday April 01, 2015 @01:22PM (#9305215)
Yep this can lead to some confusion, but the magic of OSS software is that you can download it and check :-)
Re:Redis 3.0.0 is out by zymhan on Wednesday April 01, 2015 @06:15PM (#9307340)
Won't someone please think of the childr...er, developers!
Re:Redis 3.0.0 is out by beat on Thursday April 02, 2015 @12:33PM (#9311747)
After many years of DevOps and configuration management, I've learned to think of developers as children...
Re:Redis 3.0.0 is out by simon_weber on Wednesday April 01, 2015 @01:57PM (#9305500)
Does anyone have more details on what the "embedded string" object encoding is and what workloads it helps with? The closest thing I can find is https://github.com/antirez/redis/issues/543, which seems pretty old.
Re:Redis 3.0.0 is out by antirez on Wednesday April 01, 2015 @02:08PM (#9305579)
Hey, it's very straightforward. Normally you have something like the redis object structure which has a type field, and a pointer to the actual representation of the object. If type is REDIS_STRING, you have the pointer to an "sds" string (where sds is the name of the string library used).

Now with embedded strings there is a special kind of string objects that instead use a single allocation for both the object structure and the string itself. This is slightly more memory efficient, but especially, improves memory locality a lot, so basically everything uses string objects (string types, or aggregate data types large enough to use string objects as values of the collection), will perform better.

Those special strings are used only for small strings (the majority in most work loads).

Re:Redis 3.0.0 is out by napsterbr on Wednesday April 01, 2015 @01:12PM (#9305140)
That was a nice timing, just yesterday I hacked around asyncio-redis package to provide clustering support.

Anyone interested in trying it out, or contributing, you can see the project here: https://github.com/renatomassaro/asyncio-redis-cluster

Edit: as for blocking requests, redis-py-cluster is an awesome lib that provides cluster support over the traditional redis-py client.

Re:Redis 3.0.0 is out by oimaz on Wednesday April 01, 2015 @03:23PM (#9306144)
I really like antirez approach to building system software. Build it step-by-step, iterate and very soon you have an amazing piece of software
Re:Redis 3.0.0 is out by jessaustin on Wednesday April 01, 2015 @11:28AM (#9304350)
Re:Redis 3.0.0 is out by lamby on Wednesday April 01, 2015 @01:23PM (#9305223)
(Or Debian unstable in an hour or so...)
Re:Redis 3.0.0 is out by flowerpot on Wednesday April 01, 2015 @12:34PM (#9304831)
Does the cluster feature deprecate redis-sentinel?
Re:Redis 3.0.0 is out by antirez on Wednesday April 01, 2015 @12:40PM (#9304887)
No... Sentinel will still be developed alongside with Redis Cluster. For single instance deployments where all you want is HA, Sentinel may be a more obvious way to get it compared to running Redis in cluster mode. In the long term, with plenty of early warnings, we may support the current Sentinel use case with Cluster and merge the two stuff.
Re:Redis 3.0.0 is out by flowerpot on Wednesday April 01, 2015 @12:48PM (#9304952)
Neat! Really looking forward to that!
Re:Redis 3.0.0 is out by mrmondo on Wednesday April 01, 2015 @06:13PM (#9307323)
Came here to ask this too, Sentinels are pretty awful IMO
Re:Redis 3.0.0 is out by devy on Wednesday April 01, 2015 @01:05PM (#9305089)
On https://github.com/antirez/redis/tree/3.0 branch, it shows "This branch is 1004 commits ahead, 1252 commits behind unstable".
Re:Redis 3.0.0 is out by martin1985 on Wednesday April 01, 2015 @11:22AM (#9304310)
What is the best alternative for redis?
Re:Redis 3.0.0 is out by antirez on Wednesday April 01, 2015 @12:49PM (#9304959)
Depends on the use case, but basically, there is a big set of problems you can solve with different technologies depending on your exact details, one will be better in one way, one will be better in some other way. The sensibility of the programmer is to pick the right one, in an effort to maximize the different aspects: data model fitness for the problem at hand, operational aspects, consistency guarantees, performances (number of nodes needed), scalability, simplicity (do I need support since it's a complex stuff?), and so forth.
Re:Redis 3.0.0 is out by thedufer on Wednesday April 01, 2015 @11:46AM (#9304481)
That would depend pretty strongly on what about redis isn't working for you.
Re:Redis 3.0.0 is out by joeblau on Wednesday April 01, 2015 @01:09PM (#9305127)
http://memcached.org/ used to be. I haven't done system architecture in about 2 years but when we were looking at in memory databases, it came down to Redis or Memcache.

Depending on what you need your database for, there are some that perform better than others. You can check this site for comparisons: http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

Re:Redis 3.0.0 is out by astrodust on Wednesday April 01, 2015 @02:55PM (#9305934)
Memcached is just a stupid key-value store, and by stupid I mean it just stores and retrieves values. It's extremely primitive compared to Redis.

It's not even close to the same thing as Redis except superficially.

The key/value part of Redis is just the beginning. The values themselves can be of several different types that allow for a lot more flexibility in how you store and query data.

Re:Redis 3.0.0 is out by BurningFrog on Wednesday April 01, 2015 @03:24PM (#9306156)
Redis is normally not in memory. Everything is written to disk and survives a power failure.

Memcached is just a cache. Don't store anything there you can't lose.

Re:Redis 3.0.0 is out by tim333 on Thursday April 02, 2015 @03:03AM (#9309036)
Not sure that's quite right. From the docs:

>In order to achieve its outstanding performance, Redis works with an in-memory dataset. Depending on your use case, you can persist it either by dumping the dataset to disk every once in a while, or by appending each command to a log.

Re:Redis 3.0.0 is out by BurningFrog on Thursday April 02, 2015 @11:54AM (#9311480)
Well, it's written to disk either way. You're right that if you do the "once in a while" setup, you can lose some data in a power failure.

At a previous job people started using Redis thinking it was a fast in-memory data store. It turned out we had accumulated tens of thousands of records.

I haven't measured, but I doubt it's much faster than Postgres. It does have other nice features. I like using the expiring records for caching.

Re:Redis 3.0.0 is out by JonnieCache on Wednesday April 01, 2015 @11:43AM (#9304460)
There isn't really one. Memcached fulfills a subset of its simplest use cases.
Re:Redis 3.0.0 is out by vasquque on Thursday April 02, 2015 @04:57AM (#9309389)
Hey, I also think you have to try http://tarantool.org if you wish to use persistent and an application server.
Re:Redis 3.0.0 is out by orangesareok on Wednesday April 01, 2015 @11:45PM (#9308558)
Are Oracle Coherence and Infinispan considered alternatives to redis? Don't know much about any of these besides their names to be honest.
Re:Redis 3.0.0 is out by dsetrakyan on Saturday April 04, 2015 @09:06PM (#9322746)
I think yes. Take a look at Apache Ignite Data Grid as well.
Re:Redis 3.0.0 is out by espeed on Wednesday April 01, 2015 @12:59PM (#9305043)
Look at HyperDex http://hyperdex.org/
Re:Redis 3.0.0 is out by hydrogen18 on Wednesday April 01, 2015 @01:58PM (#9305503)
Last time I looked into HyperDex it requires proprietary extensions to get access to all of its features.
Re:Redis 3.0.0 is out by misframer on Wednesday April 01, 2015 @03:50PM (#9306329)
That would be Warp[0], which adds ACID transactions across multiple keys.

[0] http://hyperdex.org/solutions/

Re:Redis 3.0.0 is out by danneu on Wednesday April 01, 2015 @07:34PM (#9307719)
One alternative is to use the database you're already using (like Postgres) until you actually need Redis.
Re:Redis 3.0.0 is out by rakoo on Wednesday April 01, 2015 @01:55PM (#9305473)
What do you want to do ?
Re:Redis 3.0.0 is out by bayesianhorse on Wednesday April 01, 2015 @12:40PM (#9304884)
April 1st is one of the worst dates for announcements ever...
Re:Redis 3.0.0 is out by Walkman on Wednesday April 01, 2015 @01:04PM (#9305079)

    p.s. no April fools here. 1 year ago I released HyperLogLog, today 3.0.0, just to contrast with some shit done this April fool lameness.
https://twitter.com/antirez/status/583279481453936640
Re:Redis 3.0.0 is out by nacs on Wednesday April 01, 2015 @01:42PM (#9305378)
Agreed, I thought it was going to be a spoof of the MongoDB 3.0 announcement/release..

http://www.mongodb.com/blog/post/announcing-mongodb-30

Re:Redis 3.0.0 is out by buster on Wednesday April 01, 2015 @11:03AM (#9304178)
Why not just linking to the google groups post or the release notes?
Re:Redis 3.0.0 is out by xai3luGi on Wednesday April 01, 2015 @12:59PM (#9305049)
Still no SSL, so using redis-client still just spews your password out all over the internet.
Re:Redis 3.0.0 is out by thezilch on Wednesday April 01, 2015 @02:02PM (#9305534)
There are plenty of alternatives to every library having to have yet another probably broken security layer. Probably better to focus on this layer being separate from everyone having to implement it.

Like, stunnel: https://www.stunnel.org/index.html and how to setup (Re: MySQL over stunnel) http://linuxgazette.net/107/odonovan.html

Re:Redis 3.0.0 is out by untog on Wednesday April 01, 2015 @01:06PM (#9305104)
Why are you connecting to a Redis box across the internet? There's a great (and after Heartbleed, prophetic) post on the Varnish web site about why they don't implement SSL, I imagine Redis would be similar:

https://www.varnish-cache.org/docs/trunk/phk/ssl.html

Re:Redis 3.0.0 is out by andrewguenther on Wednesday April 01, 2015 @01:23PM (#9305226)
I love this post. Not every single piece of software needs to include SSL support out of the box. Sometimes, for the exact reasons Varnish explains, it just doesn't make sense.
Re:Redis 3.0.0 is out by EugeneOZ on Wednesday April 01, 2015 @02:25PM (#9305709)
Varnish is other thing. For cross-datacenter replication you will want SSL. So for Redis Cluster it's a necessary thing.
Re:Redis 3.0.0 is out by danudey on Wednesday April 01, 2015 @03:07PM (#9306021)
For cross-datacenter replication you should be using a secured VPN anyway.
Re:Redis 3.0.0 is out by EugeneOZ on Wednesday April 01, 2015 @04:38PM (#9306688)
No, if your db can use SSL, then additional layer of complexity is not required.

upd.: don't get me wrong, Redis is my favorite DB, really. But better to be objective.

Re:Redis 3.0.0 is out by thedufer on Wednesday April 01, 2015 @05:15PM (#9306962)
It's hard to imagine every service in your infrastructure implementing SSL would be more secure than a single VPN tool. You are very optimistic about the difficulties of getting security right.
Re:Redis 3.0.0 is out by EugeneOZ on Wednesday April 01, 2015 @05:53PM (#9307198)
It's really simple to imagine and I even have implemented it :) "One single VPN" may (and will) fail sometimes, so count your complexity and stability with and without one extra service.
Re:Redis 3.0.0 is out by thedufer on Thursday April 02, 2015 @11:03AM (#9311078)
I'm sorry to be skeptical, but when a random person on the internet claims to have implemented SSL more securely than open source tools that are completely built around security, I tend to not believe it.

Implementing SSL is easy. Implementing SSL correctly is very difficult, and you probably won't find out you did it wrong for a long time, if ever.

Re:Redis 3.0.0 is out by EugeneOZ on Thursday April 02, 2015 @06:59PM (#9313701)
I'm not implementing SSL, I just use it. With MySQL you can just use it. With Redis you have to use VPN with all costs of VPN. Please calm down and stop forcing your preference of VPN as the only right way.
Re:Redis 3.0.0 is out by poooogles on Wednesday April 01, 2015 @05:32PM (#9307078)
Surely you would keep this in a private network? Layer 2/Encrypted VPNs?
Re:Redis 3.0.0 is out by EugeneOZ on Wednesday April 01, 2015 @06:00PM (#9307242)
For Redis - yes. But VPN is additional latency and additional service you have to monitor/restart/duplicate.
Re:Redis 3.0.0 is out by sandstrom on Wednesday April 01, 2015 @04:08PM (#9306489)
Transport security is mostly better implemented via ipsec (a VPN tunnel).

I'm happy that redis doesn't implement SSL, it just shows that they are prioritizing relevant features.

Re:Redis 3.0.0 is out by edgan on Sunday April 05, 2015 @01:18PM (#9324669)
As a operations person, this is the wrong way to go. The VPN becomes a single point of failure. Attempts at HA fail in my experience.

Also solutions like stunnel create a separate process that has to be managed. If I have one for redis, and then one for something else it is harder to tell them apart, because both will be named stunnel.