News for hackers. Stuff that matters.
Re:Tracking Changes in Directories with Python by yummyfajitas on Friday December 13, 2013 @04:30AM (#6899713)
I can't tell exactly what the goal is, but inotify might also be a simpler solution.

(Specifically, if the goal is to monitor changes as they happen and the service can be assumed to be continually running.)

Re:Tracking Changes in Directories with Python by arethuza on Friday December 13, 2013 @04:33AM (#6899721)
And the command line inotifywait - which I've used from Python to monitor changes to directories. There is even a Windows port:

https://github.com/thekid/inotify-win

Re:Tracking Changes in Directories with Python by icebraining on Friday December 13, 2013 @04:36AM (#6899727)
pyinotify is also pretty simple to use: https://github.com/seb-m/pyinotify
Re:Tracking Changes in Directories with Python by arethuza on Friday December 13, 2013 @04:48AM (#6899763)
Thanks, I was after something that would work from Python on Linux and Windows and calling out to inotifywait seemed to work well for that - this is for my home grown RESTful Dropbox-lite application which is sitting at 95% complete...
Re:Tracking Changes in Directories with Python by rytis on Friday December 13, 2013 @06:11AM (#6899993)
Or even use incrond [0], and get it call your script that does whatever needs to be done with the modified files.

[0] http://linux.die.net/man/8/incrond

EDIT: As a use case example I use this to detect when a new cert request is made to my Puppet master server. Once a CSR file is created, I check if the host is created by our provisioning system, and if so, then sign it.

EDIT2: This will eventually go into 3.4.0 - https://projects.puppetlabs.com/issues/7244

Re:Tracking Changes in Directories with Python by nodata on Friday December 13, 2013 @09:06AM (#6900495)
How does the puppet master get the csr unless the host has been provisioned?
Re:Tracking Changes in Directories with Python by rytis on Wednesday December 18, 2013 @02:37AM (#6926334)
The host is provisioned using some other automation tool, then when it comes up, it runs puppet agent. Puppet agent makes cert sign request (at which point csr file is created on puppet server).

Puppet server doesn't have this host in autosign.conf, so doesn't automatically sign the cert. And this is where incron kicks in and runs my script. The script queries the provisioning tool, sees that the host is in there, and runs puppet cert sign command.

Re:Tracking Changes in Directories with Python by tsileo on Friday December 13, 2013 @04:33AM (#6899723)
The service is not continually running, I use this method to make incremental backups with archives stored on AWS Glacier and meta-data stored on S3 (the index is stored on S3, and I can't access files on Glacier to compute deltas).
Re:Tracking Changes in Directories with Python by cmsd2 on Friday December 13, 2013 @06:13AM (#6899997)
I've been thinking about the best way to do this, and I too didn't want to rely on having a backup script running all the time.

Windows seems to have a low-level api for querying the filesystem/vfs for changes since you last looked: http://msdn.microsoft.com/en-us/library/windows/desktop/aa36...

And BTRFS has some ability to do this with find-new: http://www.tummy.com/blogs/2010/11/01/fun-with-btrfs-what-fi...

It's nice that btrfs has these new interesting features, also see send/receive, but it's not available in the vfs, and I suppose never will be.

Re:Tracking Changes in Directories with Python by xradionut on Friday December 13, 2013 @08:24AM (#6900341)
The Windows solution would seem to only work if you have admin access to the specific server or the rights to access the journal.
Re:Tracking Changes in Directories with Python by aray on Friday December 13, 2013 @04:38AM (#6899733)
I'm not sure what you mean by continually running, but inotifywait is basically just waiting on an event. As long as the process sticks around it doesn't have to do anything until it gets an inotify.
Re:Tracking Changes in Directories with Python by tsileo on Friday December 13, 2013 @04:43AM (#6899747)
I don't want to have a running process just for this, I want to be able to take a "snapshot"/"index", store some data on Glacier, and the index on S3, and later, given the index, be able to compute deltas without accessing the full archive store on Glacier.
Re:Tracking Changes in Directories with Python by blueblob on Friday December 13, 2013 @10:03AM (#6900793)
Is there a reason that you want it in python? Couldn't you do this with diff -q if you only want to know what data has changed? and then use diff to get the deltas?