I think there needs to be some way to store scripts on the server. I like the CouchDB method of a storing a view as a special document. Could use a Hash to store the code along with metadata (version, last modified, etc).

That said, if Redis can EVAL a script stored as a String or a Hash->Key, the client can deal with registering and versioning scripts.

Awesome work btw - been looking forward to scripting in Redis for a long time

Have a look at the bottom of the article, antirez suggests that a EVALSHA command might be implemented that would allow specification of a script by a SHA1 hash.

Seems like a good idea to get the rest of it sorted before deciding on that bit though.

That's what I was thinking as well. The question is, can redis (or lua, I guess) eval a script stored as a string? I imagine you would issue a redis EVAL where the script would be the Lua "eval(mykey)", is that possible?

Basically, 1) does Lua have an "eval" statement and 2) can Redis' EVAL read keys (I assume it can, since that's pretty much what the post says)?

1) yes, it has, loadstring() turns a string into an anonymous function which can then be called at will. 2) yes

So it seems you can set up your own stored procedures.

Tokyo Tyrant (and Kyoto Tyrant) have a pretty nice mechanism for doing this, http://fallabs.com/tokyotyrant/spex.html#luaext

Basically you give it a "-ext /path/to/waffles.lua" arg at startup to define a bunch of user defined functions.

This is the obvious choice, but we are in the cloud now ;) I feel like it's better if all the instances are the same regardless of the configuration. That's the whole point for not taking this path.

Even in the magical cloud, you still have configuration -- listening port at a minimum. Cloud does not remove need for deployment management, as a rule.

Why do you argue against someone who's set to go the extra-mile to make life easier for us?

Inflexible configuration mechanisms are one of the top pain points in cluster-environments (cloud or not). Kudos to antirez for realizing that and working against it.

I don't see how your argument follows, but... there is nothing "inflexible" about that setup, imho. I think antirez has a pretty good setup, but without poking around for edges, you won't find the edges :-)

Okay, learning more about it via another channel, I am pretty convinced of the path you took :-)

Two things: 1) I think this is really cool, especially the guarantee that your script is executed with the same atomicity as an internal command. That gives you a lot of freedom to write simple stuff in a simple way without too much worrying about concurrency (the INCREX sample on the linked page is a good example of what I mean - the data doesn't change between the "exists" and the "incr" call).

and 2) this opens a whole new playground for NoSQL injections: I don't know a lot (well. anything) about lua embedding, but I could imagine that these lua scripts somehow could get access to the file system. This means that you have to be really, really careful not to expose redis to the outside world and not to have it execute injected commands.

The fact that it allows separate arguments should make injections vulnerabilities a lot less likely to happen by accident since there shouldn't be any need to interpolate you arguments into the code string (this would also mess up the compiled function cache so you definitely want to do that anyway).

Also, LUA is a very small at core and I don't think it by default includes things like file system access. You pretty much just get what the host applications gives you, which in this case seems to be just the Redis API.

First, it's Lua. Not LUA.

Secondly, it does include filesystem access by default, and right now there is no decent protection against it. Hopefully, sandboxing will be implemented before this reaches anything related to stable.

You seem to be correct on both of those points.

Sandboxing Lua does seem quite easy though: http://stackoverflow.com/questions/1224708/how-can-i-create-...

Thanks almost, 100% agree, just you said it in a better English :)

Would this work like a stored procedure in e.g. MySQL, though with the disadvantage that it will not actually be stored within Redis and the advantage that it uses a more powerful language, Lua?

It looks that way. I've been playing with it and I've found two ways to actually store and reuse scripts though:

1) Use loadstring() to compile and run a script from a Redis key. EVAL "return loadstring(redis('get', KEYS[1]))()" 1 foo

2) Set a function as a global value, and call that. EVAL "myfunc = function() return 42" 0 EVAL "return myfunc()" 0

antirez said in his previous post (http://antirez.com/post/redis-and-scripting.html) that storing scripts has some problems with regard to Redis Cluster, though.

the lua api of redis seems a bit un-lua like, redis.exists('key') would be better also, returning a boolean for exists instead of a number, but it's such a small nit-pick that it hardly wants mention

the possibilities this brings is quite huge tho, so quite excited

I think everything should be observed from the point of view of an API designed to implement Redis commands, it is not a an API to access Redis from Lua in actual Lua programs.

So the whole point is to expose Redis in Lua in a way that is the most native related to the internals of Redis, and not the most comfortable/high-level.

I quite like that the Lua API is a single function, because it means it completely forward compatible already - the Lua API won't have to have new methods added to it when Redis gets new commands.

Exactly, it also means: scripting engine completely decoupled from all the rest. Also it is much simpler to do things like dynamic execution of commands, redis(my_command,"var")... or alike.

But it is just as easy to do that will a full userdata in lua. You just hook up the newindex metamethod. Even more powerfull

This seems like a really good reason, especially with Redis's focus on code quality and developer fun. That scripting.c is only 382 lines long is very cool :)

I cannot agree with that, it's just as easy to add new methods as to learn to pass a new string as the first argument. How is it more "forward compatible"?

When Redis adds a new command in the future, the Lua API code will already work with that new command without needing any modifications - hence forward compatible.

You could implement a forward-compatible database function table like this:

  local function accesscommand (t, key)
    local f = function (...)
      return redis(key, ...)
    end
    t[key] = f
    return key
  end
  db = setmetatable({}, {__index = accesscommand})
  function db.hmset (key, values)
    local t = {}
    for key, value in pairs(values) do
      t[#t + 1] = key
      t[#t + 1] = value
    end
    redis("hmset", unpack(t))
  end

Any command without a function defined in the table would create a closure on the fly that calls that command like normal. (It also stores the closure in the table to speed up future calls.) Commands that could be implemented more elegantly using Lua semantics (like HMSET) have overrides defined in the db table directly, which prevents the closure-generating function from being called.