News for hackers. Stuff that matters.

OpenKeyval.org, a free data store/social experiment

Howdy, HN'ers; A few weeks ago, we had an idea for a fun side project: a sorta-Memcache-like data storage system that could be trivially used for building simple web apps. After a few evenings of hacking, @dustball and I are pleased to present OpenKeyval.

OpenKeyval is a completely open key-value data store, exposed as a drop-dead simple web service. The goal is to make this a very easy way to persist data in web applications.

http://openkeyval.org/

As a data storage system, it's pretty basic: you can store any data you want under a particular key, and then fetch it later. You can also serve it back with a particular MIME type, so it can be used to serve small resources. The keyword small is operative here: the data is limited to 64 KiB, which is plenty of space for storing interesting data, but not really big enough for storing images or any other sort of media.

As a social experiment, I think it's sort of fun: there's no authentication of any kind -- just come up with sufficiently unique keys to make it unlikely that anyone else will stumble upon them. Scouts honour that anything you put here is safe in our hands.

I'm curious as to what people think this might be useful for, and if you have any feature requests. I'd love to see people try to build something with it -- but go easy on us, since it's just an experiment. :)

( 24 comments | 46 points | on HN )
Re:OpenKeyval.org, a free data store/social experiment by aaronblohowiak on Friday October 29, 2010 @01:31AM (#1846279)
Clickable: http://openkeyval.org/

If you want to help people make webapps, promote the jsonp store to the top of the documentation and explanation of how it is used (including the fact that you aren't going to be able to store anywhere near 64kib with it -- more like ~2kb). While it is less neat, it seems to be the only viable solution for todays non-IE limitations on cross-site form posting.

Re:OpenKeyval.org, a free data store/social experiment by markchristian on Friday October 29, 2010 @01:01AM (#1846213)
Oh, I forgot to mention -- it's open source, too. http://github.com/shinyplasticbag/openkeyval
Re:OpenKeyval.org, a free data store/social experiment by rlpb on Friday October 29, 2010 @08:56AM (#1847105)
Have you considered supporting the memcached protocol? It might help with takeup as client libraries already exist.

http://github.com/memcached/memcached/blob/master/doc/protoc...

Re:OpenKeyval.org, a free data store/social experiment by markchristian on Friday October 29, 2010 @11:36AM (#1847631)
We have discussed it, and even considered just building an HTTP layer on top of it, but haven't done anything with it yet. As for supporting the memcached protocol, we considered that, too, and at this point, we're not targeting that use case. Personally, I've been planning on using this for entirely client-side JavaScript apps, where a memcache layer wouldn't be useful at all.

(That being said, the code is up on github, and patches are welcome. :)

Re:OpenKeyval.org, a free data store/social experiment by thepumpkin1979 on Friday October 29, 2010 @02:57AM (#1846436)
Interesting, but maybe you can create some sort of namespaces so we don't override each other's keys.
Re:OpenKeyval.org, a free data store/social experiment by markchristian on Friday October 29, 2010 @01:44PM (#1848161)
You can create your own namespaces by prefixing your keys. The whole idea of this is a flat, totally open namespace.
Re:OpenKeyval.org, a free data store/social experiment by stagas on Friday October 29, 2010 @10:40AM (#1847435)
I'm using a similar approach in a node.js module if you'd like to try it. http://github.com/stagas/chaos stores the data in files using the sha1 hash key like you do, you could easily implement an http api layer on top of it. I get about 2k writes/s and 4k reads/s on a regular hd, and 40k-60k writes/reads per second on a ssd.
Re:OpenKeyval.org, a free data store/social experiment by naz on Friday October 29, 2010 @03:16AM (#1846465)
Nice idea, super simple, and poses some interesting questions.

There could be security issues with this given you can serve arbitrary MIME types. XSS won't be a problem because there are no cookies to steal. Maybe you could make a script that self-propagates across the key-space using AJAX.

Re:OpenKeyval.org, a free data store/social experiment by acme on Friday October 29, 2010 @04:52AM (#1846644)
Reminds me of OpenDHT, "a publicly accessible distributed hash table (DHT) service", which has since been taken down: http://www.opendht.org/ I do think being able to store information without too much hassle is pretty useful.
Re:OpenKeyval.org, a free data store/social experiment by pudquick on Friday October 29, 2010 @06:43AM (#1846817)
... which in turn reminded me of the Vanish project http://vanish.cs.washington.edu/research.html which leveraged both OpenDHT and Vuze's (aka Azureus) DHT to store data (specifically: redundantly stored encryption keys, with the intent of letting them expire).
Re:OpenKeyval.org, a free data store/social experiment by wazoox on Friday October 29, 2010 @07:18AM (#1846873)
Instead of storing the data in files, this looks like a perfect use case for TokyoCabinet or similar storage engines.
Re:OpenKeyval.org, a free data store/social experiment by thepumpkin1979 on Friday October 29, 2010 @03:09AM (#1846455)
Share your keys on Twitter: #openkeyval
Re:OpenKeyval.org, a free data store/social experiment by markchristian on Friday October 29, 2010 @11:42AM (#1847657)
Nice. :)
Re:OpenKeyval.org, a free data store/social experiment by bhiggins on Friday October 29, 2010 @01:49AM (#1846309)
hey sorry to burst your bubble but this is pointless.
Re:OpenKeyval.org, a free data store/social experiment by aaronblohowiak on Friday October 29, 2010 @03:29AM (#1846492)
This post is a great example of being "a hater". You aren't disagreeing about the application of a tool to a problem, you are dismissing all opportunity for anyone to enjoy or get value out of the item in question. Since the creators put effort in to creating this, it at least has value to them. So do the people that have upvoted it. So you're dismissing their opinion without offering any information, which is not very helpful.
Re:OpenKeyval.org, a free data store/social experiment by StavrosK on Friday October 29, 2010 @07:38AM (#1846918)
I share the GP's feeling, although not his short-sightedness. I would have made the same comment in a different way, namely:

I have no use for this and can't imagine where I would, so I can't give you any feedback. However, I'm sure that some people will find it useful and dream up new things to do with it, so good job!

EDIT: Actually! I think that this would be a great way to persist data in static, local html documents. You can probably create a full-blown web app with this in just HTML which someone can unzip and run in their browser.

Re:OpenKeyval.org, a free data store/social experiment by markchristian on Friday October 29, 2010 @01:46PM (#1848166)
Your EDIT hit the nail on the head — that's exactly what I had in mind when I wrote this. I plan on writing some web apps that can run from anywhere as a pile of static files, using this as their backing store.
Re:OpenKeyval.org, a free data store/social experiment by bhiggins on Friday October 29, 2010 @03:32AM (#1846498)
fine. here's some information, then: you're promoting cargo cult thinking.
Re:OpenKeyval.org, a free data store/social experiment by markchristian on Friday October 29, 2010 @02:02AM (#1846332)
Nothing is pointless! Everything is an adventure. :)
Re:OpenKeyval.org, a free data store/social experiment by heretoo on Friday October 29, 2010 @02:35AM (#1846397)
Exactly. I was trying to think how to build a chat program on this, and establish keys between two chatees for communicating, but it seems impossible to make a public channel/key for registering interest in chatting in the first place, without using an untrusted readwrite channel/key.

An append only store might solve it, with a key that is globally known. Some sort of broadcast channel. An append only, with limited size appends, and cleared at regular intervals.

Re:OpenKeyval.org, a free data store/social experiment by markchristian on Friday October 29, 2010 @01:49PM (#1848185)
Good challenge! Here's a start:

Say you and I want to chat with each other. We both pick a random key to store our messages in. Let's call those key[markchristian] and key[heretoo]. Each of those has a read-only version -- readonly[markchristian] and readonly[heretoo]. I'll post the key readonly[markchristian] to "markchristian-to-heretoo". You post the key readonly[heretoo] to "heretoo-to-markchristian". Our chat clients can just poll those keys, looking for new messages -- whatever hand-wavy format we want to have.

Problems: 1. Requires us to agree to chat ahead of time and know each other's usernames. 2. Nothing to stop a malicious third-party from putting a different readonly key at the "heretoo-to-markchristian" pointer keys.

Re:OpenKeyval.org, a free data store/social experiment by bhiggins on Friday October 29, 2010 @03:19AM (#1846474)
your enthusiasm is infectious in a bad way.
Re:OpenKeyval.org, a free data store/social experiment by konad on Friday October 29, 2010 @05:08AM (#1846670)
I'm with you bhiggins
Re:OpenKeyval.org, a free data store/social experiment by kh411d on Friday October 29, 2010 @04:42AM (#1846624)
hmm...