News for hackers. Stuff that matters.
Re:Show HN: A versioned filesystem inspired by Git by tolmasky on Tuesday August 28, 2012 @12:23PM (#4444472)
OK, first off -- awesome! Secondly, I have a number of questions that probably have to do with the fact that I'm unfamiliar with a lot of this stuff.

1. Would it be possible to build this fully on top of git (perhaps using libgit2)? I ask, because the "holy" grail for me is finding some magic way for my designers to use git without knowing it. Right now my designers (and I think most non-programmer people), love to use Dropbox. Dropbox has a number of issues as a shared project tool in my perspective (not the least of which being that you are only allowed ONE dropbox account per computer, so you start having a shared folder mess and your dropbox begins to balloon in size on every computer with that account). It appears that something like this could look like Dropbox to everyone disinterested in vcs and act like Git to everyone else who cares.

2. The other reason for libgit2 is to ideally also git-push after every edit/save, to something like github.

3. Max OS X support please :)

Re:Show HN: A versioned filesystem inspired by Git by artagnon on Tuesday August 28, 2012 @12:38PM (#4444550)
1. I only use (a modified version of) packfile/ packfile idx from Git. No commits are actually created. It sounds like you're looking for something like flashbake (http://bitbucketlabs.net/flashbake/).

2. phoenixfs doesn't create commits, so no git-push.

3. I currently don't have access to an OS X box, sorry :(

Re:Show HN: A versioned filesystem inspired by Git by s_tec on Tuesday August 28, 2012 @03:04PM (#4445300)
Have you tried SparkleShare? It sounds exactly like what you want. It connects to a normal Git server and presents the repository as a Dropbox-esque folder. It has support for Linux, Windows, and Mac, although it's mainly a Linux project.
Re:Show HN: A versioned filesystem inspired by Git by sciurus on Tuesday August 28, 2012 @06:01PM (#4446109)
More info:

"SparkleShare is a 100% free and open source, cross-platform file sharing system started by Hylke Bons. The idea for the project came from the 2010 GNOME UX Hackfest. It's very similar to Dropbox or Box.net, but it is backed by the free and open source version control system git.

Basically, the way it works is that you will have a special folder on your computer, called 'Sparkleshare', and everything in that folder is also present in a version-controlled remote git repository. If you create a new file in a Sparkleshare folder, it will immediately be committed and pushed into the remote git repository, and everyone else who is using Sparkleshare to connect to that folder will be able to see the file you just created appear in their Sparkleshare folders. If someone else opens the file you just created and modifies it, their Sparkleshare client will automatically commit and push their changes to the repository, and you'll be able to see them in your Sparkleshare folder (and you'll get a handy notification!)

Sparkleshare is a great, free & open source way to collaborate on files, without worrying about manual version control or having to manually upload and broadcast links to your work."

http://sparkleshare.org/

Re:Show HN: A versioned filesystem inspired by Git by tolmasky on Tuesday August 28, 2012 @11:13PM (#4446901)
Wow I will definitely take a look at this.
Re:Show HN: A versioned filesystem inspired by Git by martythemaniak on Tuesday August 28, 2012 @12:29PM (#4444502)
Check out Pixelapse, which is kind of like github for designers: http://www.pixelapse.com/
Re:Show HN: A versioned filesystem inspired by Git by tolmasky on Tuesday August 28, 2012 @12:35PM (#4444534)
Yeah I asked for an invite ages ago, haven't heard back. if they ever let me in it might do the trick.

Separately, actual github for designers would be cooler because I could:

1. Have an "artwork" git repo which my designers interact with in this way, but which I include as a submodule to a larger git repo for my programmers.

2. Use the same bug tracking for art as I do programming

3. Backup the git repo because its an open protocol vs. some proprietary thing.

Again though, I'm lazy and don't have time to write project management tools for myself so if these guys ever got around to sending me an invite I'd probably just settle.

Re:Show HN: A versioned filesystem inspired by Git by debacle on Tuesday August 28, 2012 @03:24PM (#4445434)
The windows github client is relatively idiot proof. I don't know if they offer something similar for Macs.
Re:Show HN: A versioned filesystem inspired by Git by zerovox on Tuesday August 28, 2012 @06:52PM (#4446287)
They do, it's much nicer in fact. http://mac.github.com/
Re:Show HN: A versioned filesystem inspired by Git by RKearney on Tuesday August 28, 2012 @08:44PM (#4446572)
The Github app for Macs is just too bare in my opinion.

With that being said, I use SourceTree[1] every day and love it.

[1]http://www.sourcetreeapp.com/

Re:Show HN: A versioned filesystem inspired by Git by akg on Tuesday August 28, 2012 @10:24AM (#4443781)
Congrats on a tremendous technical feat!

Any advantage of using a full-scale FS over something like git-annex (http://git-annex.branchable.com/)?

Re:Show HN: A versioned filesystem inspired by Git by artagnon on Tuesday August 28, 2012 @10:48AM (#4443903)
The seamless workflow that only a full-blown filesystem can offer- new revisions are created automatically when files are edited and saved.
Re:Show HN: A versioned filesystem inspired by Git by fuzzix on Tuesday August 28, 2012 @11:15AM (#4444081)
I like it. It reminds me of VMS's versioning.

I implemented something like VMS style versioning with Fuse and Perl for a talk/demo a couple of years ago - it simply created a new version on open() if any writeable flags were passed - not robust, but functional enough to demo.

edit Possessed by VMS, not multiple VMSes.

Re:Show HN: A versioned filesystem inspired by Git by spartango on Tuesday August 28, 2012 @03:32PM (#4445464)
You should be able to achieve this with Filesystem events (inotify/kqueue/filesystemwatcher). Just another way of doing things.

I'm not the biggest fan of FUSE filesystems(until highly-tuned, ala NTFS-3G), but they certainly do work.

Re:Show HN: A versioned filesystem inspired by Git by halayli on Tuesday August 28, 2012 @04:19PM (#4445685)
No, you cannot achieve atomicity using these calls. By the time you receive a notification the file might have been edited twice.
Re:Show HN: A versioned filesystem inspired by Git by spartango on Tuesday August 28, 2012 @05:30PM (#4445978)
This is true, but not without drawbacks:

* Using FUSE you gain the atomicity through synchronicity; when a write occurs, you stand between the disk and request. In so doing, you introduce the potential for latency (nevermind the latency inherent to FUSE).

* Using fsevents, you get notifications asynchronously, so there's no I/O performance hit. You may not get the events right away, but with fsevent APIs there's a bounded queue so you generally don't miss events, unless there are too many and the buffer is overflowed[1,2]. The events don't contain the actual file data, so you will lose intermediate edit data.

Thinking about this, it would be cool to exploit a Copy-on-Write filesystem (like ZFS) to scalably add the edit data to fsevent apis. I suppose you could preserve write buffers in memory temporarily instead of CoW, but that would have a nasty memory footprint.

[1]http://msdn.microsoft.com/en-us/library/system.io.filesystem... - Internal buffer size

[2]http://docs.oracle.com/javase/7/docs/api/java/nio/file/Watch... (see the watchkey queue and overflow)

Re:Show HN: A versioned filesystem inspired by Git by gwern on Tuesday August 28, 2012 @04:44PM (#4445809)
If the file is changing that fast, do we really care? I'd be happy to have all my files tracked at only 5s temporal resolution...
Re:Show HN: A versioned filesystem inspired by Git by rhizome on Tuesday August 28, 2012 @04:53PM (#4445848)
Sure, but which 5s? :)
Re:Show HN: A versioned filesystem inspired by Git by p9idf on Tuesday August 28, 2012 @12:45PM (#4444587)
Does your file system create revisions of the entire tree, or only revisions of individual files within the tree as VMS and RCS do?† Because there are some problems with the latter approach. One problem described by Geoff Collyer (the C News guy) is that "version numbers of related files are not kept in synch, so you can't just cp *';6' /tmp/compile".††

† I tried answering that question myself, but your program kept crashing.

†† http://9fans.net/archive/2001/11/759

Re:Show HN: A versioned filesystem inspired by Git by artagnon on Tuesday August 28, 2012 @12:54PM (#4444634)
Revisions of individual files only. Yeah, you can't do that kind of sorcery with phoenixfs.

† Fixed. Sorry about that.

Re:Show HN: A versioned filesystem inspired by Git by gcr on Tuesday August 28, 2012 @01:31PM (#4444858)
Can this handle large binary files? That's been my main problem with storing backups of my servers in git, and it's one of the strong-ish points of git-annex.

There seems to be nothing inherently limiting in Git's repository design. Interested parties can look into bup, for example, which stores large binary files in git's packfile format. https://github.com/apenwarr/bup

Re:Show HN: A versioned filesystem inspired by Git by artagnon on Tuesday August 28, 2012 @01:39PM (#4444902)
Yes; I don't see why not. I haven't stress-tested this enough to tell for certain though.
Re:Show HN: A versioned filesystem inspired by Git by tensor on Tuesday August 28, 2012 @02:20PM (#4445082)
This is relevant:

http://www.hpl.hp.com/personal/alistair_veitch/papers/elepha...

If only half the stuff done by the likes of IBM and HP were to make it into common usage the world would be a very different place.

Re:Show HN: A versioned filesystem inspired by Git by al3xbio on Tuesday August 28, 2012 @09:52AM (#4443650)
Nice. Why not use libgit2 (https://github.com/libgit2/libgit2)?
Re:Show HN: A versioned filesystem inspired by Git by artagnon on Tuesday August 28, 2012 @10:34AM (#4443820)
It's a bit of an overkill for my purposes- I've only copied out (and modified) a little of the packfile/ packfile index generation code from git.git. Plus, dealing with the actual code is a great learning exercise :)
Re:Show HN: A versioned filesystem inspired by Git by mosburger on Tuesday August 28, 2012 @11:41AM (#4444239)
Nice! Reminds me a bit of a better version of VMS's versioned filesystem.

Also: I'm old enough to have used VMS's filesystem. :(

Re:Show HN: A versioned filesystem inspired by Git by josephscott on Tuesday August 28, 2012 @01:10PM (#4444744)
Agreed, VMS filesystem was the first thing that came to mind for me as well. Seems rather sad that in 2012 we don't have that as a standard feature like we did back then.

For the record, you don't have to be all that old to have used VMS's filesystem :-)

Re:Show HN: A versioned filesystem inspired by Git by ScottBurson on Tuesday August 28, 2012 @01:08PM (#4444730)
Chill out. I'm old enough to have used ITS, which introduced versioning(†).

(† Apparently: http://en.wikipedia.org/wiki/Versioning_file_system)

Re:Show HN: A versioned filesystem inspired by Git by Hominem on Tuesday August 28, 2012 @01:17PM (#4444789)
My company still uses VMS specifically for the versioned filesystem.
Re:Show HN: A versioned filesystem inspired by Git by ylem on Tuesday August 28, 2012 @01:26PM (#4444828)
We just retired our old vax a couple of years ago :>
Re:Show HN: A versioned filesystem inspired by Git by adam-a on Tuesday August 28, 2012 @10:43AM (#4443879)
Sounds a bit like ZFS snapshots[1]. I always wished that would catch on a bit more, seems like it would be very useful.

[1]: http://en.wikipedia.org/wiki/ZFS#Snapshots_and_clones

Re:Show HN: A versioned filesystem inspired by Git by justinmk on Tuesday August 28, 2012 @12:55PM (#4444637)
Re:Show HN: A versioned filesystem inspired by Git by halayli on Tuesday August 28, 2012 @04:02PM (#4445603)
Re:Show HN: A versioned filesystem inspired by Git by voltagex_ on Tuesday August 28, 2012 @10:41PM (#4446838)
Might want to post that to the issue tracker
Re:Show HN: A versioned filesystem inspired by Git by icebraining on Tuesday August 28, 2012 @10:25AM (#4443791)
Extremely interesting. I wonder: when does it "commit"/create a new rev?
Re:Show HN: A versioned filesystem inspired by Git by artagnon on Tuesday August 28, 2012 @10:35AM (#4443828)
When you modify a file and save it.
Re:Show HN: A versioned filesystem inspired by Git by rix0r on Tuesday August 28, 2012 @11:20AM (#4444115)
So on close() ?
Re:Show HN: A versioned filesystem inspired by Git by artagnon on Tuesday August 28, 2012 @12:10PM (#4444385)
Yes. The event is called `release`: see phoenixfs_release() in fuse.c:374.
Re:Show HN: A versioned filesystem inspired by Git by ferrouswheel on Saturday September 01, 2012 @09:10PM (#4465775)
What about when a file is fsynced?
Re:Show HN: A versioned filesystem inspired by Git by nicholassmith on Tuesday August 28, 2012 @10:14AM (#4443739)
This looks pretty good. I had a look at doing something similar (but wayyyyy smaller scale than full FS), and gave up as there was many dragons. Nice to see someone pull it off.
Re:Show HN: A versioned filesystem inspired by Git by jarvuschris on Tuesday August 28, 2012 @06:51PM (#4446280)
I'm working on a similar concept which provide copy-on-write over HTTP as well: http://emr.ge
Re:Show HN: A versioned filesystem inspired by Git by jeremyjh on Tuesday August 28, 2012 @11:39AM (#4444225)
Very cool. I would like to have some automatic revisions for example of my /etc tree.
Re:Show HN: A versioned filesystem inspired by Git by RobotCaleb on Tuesday August 28, 2012 @04:27PM (#4445730)
There's a tool that will commit changes to /etc before every apt-get install.

This might be what I'm thinking of: http://joeyh.name/code/etckeeper/

Re:Show HN: A versioned filesystem inspired by Git by jblz on Tuesday August 28, 2012 @12:15PM (#4444418)
This is tremendously cool!

Any plans for MacOS (or other *nix) support?

Re:Show HN: A versioned filesystem inspired by Git by artagnon on Tuesday August 28, 2012 @12:32PM (#4444519)
There's a FUSE for OS X, so a port should be possible (should be possible for other *nix too, as long as they have FUSE). I currently don't have access to an OS X machine, so no immediate plans.
Re:Show HN: A versioned filesystem inspired by Git by prezjordan on Tuesday August 28, 2012 @10:35AM (#4443831)
Wow, awesome project!