I built one recently. We have a SQLite VFS built on top of Zookeeper for doing metadata-like stuff. We can pull between 1000-100000 reads per seconds out of it.

I realize that this is a total abuse of what Zookeeper does, and how it is supposed to work, but hey, this is a solution that's _simple_ and _correct_.

Here are the benchmarks comparing the traditional VFS with the ZK VFS: https://gist.github.com/sargun/7503ba636c1a0eb2e2f5cda7a6a23...

PS: Feel free to ask me any questions

That's the sort of thing I was interested in. Did you split the data up into kv pairs somewhat like this shim: http://www.sqlite.org/src/doc/trunk/src/test_multiplex.c ?

Or is just a small database where 1 sqlite file == 1 key?

Pretty interesting. Do you have source code you can share?

Not yet open source, and it probably wont be for a while, given it's in $EMPLOYER's proprietary product.

I looked up SQLite VFS. That's pretty interesting, though I am not sure if rqlite has sufficient control over that low a layer in the system, due to the way rqlite is built.

I opened an issue anyway, to look into this.

https://github.com/rqlite/rqlite/issues/108

I was really just curious if you had looked at it and ruled it out. Pretty sure it would be a different approach not compatible with the way rqlite works. At the VFS level, you're only seeing the reads/writes/locks to the data file...no visibility to the sql.

If you implement a VFS, you have to either implement the WAL subsystem yourself, which isn't pluggable at the moment, or go with journaling mode, which is kinda meh.

>>If you implement a VFS, you have to either implement the WAL subsystem yourself

It's pluggable in the sense that it maps the the shared memory calls for WAL to your calls. But yeah, figuring out how to emulate shared memory across a cluster is on the implementer.