Re:Introducing Drake, a kind of ‘make for data’
by blablabla123 on Thursday January 24, 2013 @03:01PM
(#5111820)
retries and email notifications is a good one. Currently I do something similar with cronjobs, rsync, shell scripts and some custom tools -- on multiple boxes. (Email notification with mailx) Works in theory pretty well, in practice race conditions become a problem, making it sometimes annoying because I need to run things manually when I need up to date processed data. If I had retries, this would be an improvement.
Re:Introducing Drake, a kind of ‘make for data’
by aboytsov on Friday January 25, 2013 @07:22AM
(#5115088)
Got ya. Please voice your opinion about the priority in which features should be implemented by submitting a feature request at https://github.com/Factual/drake/issues, or +1'ing an existing one.
There are so many potential features to be added to Drake, and a lot of them have already been thought about and spec'ed out, that we need some sort of a way to figure out what to do first.
Of course, if you'd like to actively contribute, we'd be ecstatic.
Artem.
The concept of Make is not unique. Everything that has dependencies and executes steps is similar to Make in concept. Drake is no exception, and it can be replaced with Make, but no more so than Rake, Ant or Maven can be replaced by Make. That is, if it's trivial - yes. Just a bit more complicated - no.
Some things are merely painful to implement with Make, some are just impossible:
These are just what's implemented already. Other things are planned such as: Requirements for building executables and working with large, complicated and expensive data workflows are quite visible different, and the most important thing about Drake is that it provides the platform for convenient features (such as versioning or email notifications) to be implemented. And once they are, every data workflow can take advantage of them.I guess, if Make was really, really extendable, we could have considered it as a platform for all this. But it's not, and hacking all of that into Make's source code in C would be, I'm sure, a much greater pain than writing Drake.
Artem.