News for hackers. Stuff that matters.
Re:Schedule – Python job scheduling for humans by mckoss on Saturday October 03, 2015 @09:34AM (#10323825)
I personally dislike this style of "fluent api". It is not more "for humans" just because it reads as English sentences.

In fact, it can be quite difficult to reason about. What, for example is "schedule. every(10)"? By itself, it does not seem to represent a useful concept other than a "partial" schedule statement.

I would prefer to use an api using keyword arguments to set the various (optional) parameters to build a schedule.

Re:Schedule – Python job scheduling for humans by Drdrdrq on Saturday October 03, 2015 @11:33AM (#10324221)
Exactly! One example of this is fudge... Annoying to use, especially when docs are less than perfect.
Re:Schedule – Python job scheduling for humans by StavrosK on Saturday October 03, 2015 @07:23AM (#10323477)
I've been talking with a few developer friends about libraries lately, and this one has made me realize what they're saying. As Python developers, DSLs are completely not in our DNA, but this library strikes me as something that would look much more Pythonic with a DSL.

Wouldn't it be better to have descriptions of:

Every day at 10:30 do myfunc

Every Monday do otherfunc

Every hour do whatever

Rather than the current .every().day.at("10:30").do(myfunc)?

Re:Schedule – Python job scheduling for humans by gvalkov on Saturday October 03, 2015 @08:29AM (#10323611)
I find it hard to appreciate these inventive types of API design. Something more traditional and boring, like Celery's crontab[1] or dateutils rrule[2] seem quite sufficient. For example:

   crontab(hour=18, minute=30, day_of_week='*')
[1]: http://celery.readthedocs.org/en/latest/userguide/periodic-t...

[2]: https://labix.org/python-dateutil#head-470fa22b2db72000d7abe...

Re:Schedule – Python job scheduling for humans by StavrosK on Saturday October 03, 2015 @08:33AM (#10323619)
I'm not sure why, given two equally flexible designs, we should choose the least expressive. The fact that cron is sufficient doesn't mean I don't still have to google how to run things every three hours every second Monday, for example.
Re:Schedule – Python job scheduling for humans by draegtun on Saturday October 03, 2015 @10:53AM (#10324080)
I agree. This scheduler dialect for Rebol is pretty much the DSL you describe - http://softinnov.org/rebol/scheduler.shtml
Re:Schedule – Python job scheduling for humans by anentropic on Saturday October 03, 2015 @01:22PM (#10324574)
I'd rather have neither. I particularly dislike the .every().day.at("10:30").do(myfunc) thing where methods and attributes are abused in order to artificially make it read like a sentence.

This is not Pythonic at all <troll>it's like some childish Ruby code</troll>

Re:Schedule – Python job scheduling for humans by omni on Saturday October 03, 2015 @01:51PM (#10324691)
What specifically do you think makes this un-Pythonic? It's very readable.
Re:Schedule – Python job scheduling for humans by StavrosK on Saturday October 03, 2015 @01:25PM (#10324579)
That's not a very constructive reply.
Re:Schedule – Python job scheduling for humans by St-Clock on Saturday October 03, 2015 @09:07AM (#10323735)
We prefer to use Celery to schedule periodic tasks [1] and are moving most of our cron jobs to Celery because it is easier to monitor, test, retry, and send reports on failed jobs.

We extended [2] django-celery db backend to store enough information about the periodic tasks to retry them directly in Django admin or to filter the results (we support storing the task results in JSON instead of pickle).

Celery is probably more difficult to configure but is far more flexible and robust than this lib...

[1] http://celery.readthedocs.org/en/latest/userguide/periodic-t... [2] https://github.com/resulto-admin/django-celery-fulldbresult

Re:Schedule – Python job scheduling for humans by tluyben2 on Saturday October 03, 2015 @06:42AM (#10323383)
My team likes Celery to do stuff like this always which is indeed better/necessary for tasks which need to be executed always even when web server dies/restarts. For a lot of jobs that is not needed (where the background is directly related to the foreground and without the foreground process working properly the background processes make no sense anymore) however and then Celery is a pain in the ... to setup & maintain properly and, frankly, overkill, even with the simple setup solutions like Redis. I would recommend in process schedulers for those cases as they require no setup.
Re:Schedule – Python job scheduling for humans by drudru11 on Saturday October 03, 2015 @02:01PM (#10324746)
So - this is interesting. The thing that I always look for is how the system deals with all of the time issues that come up. For example, how does it deal with Daylight Savings, or a leap second, or adjustments to time, or the daemon getting restarted, etc. I don't see anything specific to this library that handles these scenarios.

Is there any system out there that has a good design for handling these issues?

Re:Schedule – Python job scheduling for humans by lifeisstillgood on Saturday October 03, 2015 @04:17PM (#10325194)
My issue is how to make this distributed (unless Inmissed that)

Have many out of process (on disk) files that contain some or all of the phrases "start package a with Params b at time c"

And have the scheduler pass over the actual running to another process - possibly another machine entirely

Would like to get my teeth into that

Re:Schedule – Python job scheduling for humans by AnthonBerg on Saturday October 03, 2015 @09:39AM (#10323842)
I like the feel of it and the thought behind it, but ... while true, sleep?

I get very uncomfortable with any kind of polling, especially polling timers for checking for jobs with timespans between them. Maybe it's just me?

Re:Schedule – Python job scheduling for humans by est on Saturday October 03, 2015 @11:19AM (#10324184)
every cron or cron-like service has some kind of polling behind. Beanstalk, redis-queue, you name it. Many open source redis based scheduler are all implemented in a sleep(5) loop.

Even the Unix cron is implemented with a sleep() call

https://en.wikipedia.org/wiki/Cron#History

Re:Schedule – Python job scheduling for humans by drudru11 on Saturday October 03, 2015 @01:58PM (#10324728)
As another commentor noted, this is the way every system is implemented. It is simple, slightly inefficient, and deals with most of the nasty problems that arise when dealing with time on all systems today.
Re:Schedule – Python job scheduling for humans by vas1 on Saturday October 03, 2015 @07:59AM (#10323542)
Currently I use APScheduler for this exact use-case. This looks interesting and I will be taking a closer look at it.
Re:Schedule – Python job scheduling for humans by Drdrdrq on Saturday October 03, 2015 @11:46AM (#10324271)
Thank you, looks really nice and customizable! Did you have any issues with it?
Re:Schedule – Python job scheduling for humans by hotdox on Monday October 05, 2015 @05:52AM (#10330786)
Not the author of previous comment, but my main problem is inability to limit time of job execution in APScheduler.
Re:Schedule – Python job scheduling for humans by arsalanb on Saturday October 03, 2015 @07:17AM (#10323464)
Can this be used in production?
Re:Schedule – Python job scheduling for humans by est on Saturday October 03, 2015 @11:23AM (#10324192)
You'll need to wrap it in `threading` to avoid block your main thread, or use greenlet, or wrap it as a RESTful service, then you have the problem of high-availability, so a distributed shared consensus system would be better, the next thing you know you have NTP issues... The problem goes on.
Re:Schedule – Python job scheduling for humans by bpicolo on Saturday October 03, 2015 @12:26PM (#10324387)
Job scheduling not really rocket science, so sure.

Job scheduling when things fail is harder

Re:Schedule – Python job scheduling for humans by arthurcolle on Saturday October 03, 2015 @10:39AM (#10324037)
Why is the time.sleep(1) needed inside the while loop?
Re:Schedule – Python job scheduling for humans by Drdrdrq on Saturday October 03, 2015 @11:34AM (#10324232)
Otherwise app would burn CPU like crazy...
Re:Schedule – Python job scheduling for humans by est on Saturday October 03, 2015 @11:14AM (#10324161)
Because that's one second resolution of the scheduler.
Re:Schedule – Python job scheduling for humans by mpdehaan2 on Saturday October 03, 2015 @09:14AM (#10323756)
Aside: I'm not a fan of the "for humans" label that seems to be common among some Python projects now.

It implies the other tools are possibly designed by monkeys, to a degree, which is often totally not the case.

As a particular example, while some libs are "friendly", they are also limited.

In this case, celery is implied to be not "for humans", but it actually offers distributed job queuing and jobs that can survive a reboot, which is pretty darn important.

I'm not going to name names of other libs doing this, but there are a few that have picked API simplicity over complete feature set. This is obviously a balance - BUT I think it unfairly casts the alternatively libs in a questionable light, and often they can do a lot more.

FWIW, the Ruby-design into the method space can also be reduced if desired to make this a bit more Pythonic (though some things are personal preferences):

    scheduler = schedule.scheduler()

    scheduler(callback=job).every(minutes=10)

    scheduler(callback=job).every(day=1).at(hour=10, minutes=30)
Having functions like "Wednesday" or "minutes" complicate programatic usage, where in the above, it would be possible to pass in a constant.

One thing I also notice (not sure if Ruby inspired or not), is that it's keeping global state in the module (see run_pending) rather than having different scheduler objects. This somewhat limits reusability by having this embedded magic and not being able to isolate schedulers.

Re:Schedule – Python job scheduling for humans by gjm11 on Saturday October 03, 2015 @11:04AM (#10324129)
"For humans" is surely intended to be the opposite of "for computers". It's meant to say "When we designed this thing, we made it a high priority for it to be easy to learn, easy to remember, and easy to use".

There's certainly an implication that other libraries are designed without sufficient attention to human factors, and maybe that's kinda rude, but the same goes for making any positive claim. "A library for efficient high-precision computations": are all the others inefficient? "A flexible and powerful tool for retrieving structured data from web APIs": are all the others inflexible and weak? "A standard-conformant C++ parser": are all the others broken? (None of those examples, so far as I know, are from real projects, but clearly they all could be.)

Re:Schedule – Python job scheduling for humans by andreasvc on Saturday October 03, 2015 @11:18AM (#10324178)
I also find the "for Humans" thing annoying, but not for the reasons you list. The examples you list are all specific and informative. In the case of "for Humans" what is probably meant is something like "straightforward" or "simple". The thing is, there isn't anything that is NOT designed "for Humans", so the label is meaningless, IMHO.
Re:Schedule – Python job scheduling for humans by icebraining on Saturday October 03, 2015 @10:44AM (#10324048)
Monkeys? Where did that come from? The idea of "for humans" just means that the UI was developed with an humane approach - as opposed to an undesigned UI, or one designed merely for the simplicity of the implementation, and that doesn't take into consideration things like our need for consistency to make up for a non-perfect memory.

That said, I personally don't think Celery suffers from this problem, but API that was being replaced by the first project to used this moniker - urllib2 - definitively did.

Re:Schedule – Python job scheduling for humans by mpdehaan2 on Saturday October 03, 2015 @11:30AM (#10324212)
It's a stand in for something "not for humans". On second thought, monkeys would require an even easier UI. Perhaps "people that like pain".

I don't think it's fair to say the other tools weren't designed, or merely designed the easy way.

urllib2 has some rough edges - and some things should be better, but it can also do some things requests can't do, and I've had to reach for it on a few occasions. requests has basically optimized for the easy case, where

urllib2 wished to be more flexible in certain areas, so it deserves some merit there too.

Re:Schedule – Python job scheduling for humans by Drdrdrq on Saturday October 03, 2015 @11:30AM (#10324209)
> API that was being replaced by the first project to used this moniker - urllib2 - definitively did.

For curious: that was python library requests. And the claim was more than deserved in that case.

Re:Schedule – Python job scheduling for humans by jimmytucson on Saturday October 03, 2015 @12:27PM (#10324390)

    > "for humans"
The earliest example of this that I could find is from a 2007 article by Jeff Atwood entitled "Sorting for Humans : Natural Sort Order". Coincidentally, Atwood double-dipped on it a few months later with an article called "Monkeypatching for Humans" (but I think he meant "monkey" in a different way).

I think in recent years, Kenneth Reitz has really taken "for humans" and run with it: "Python for Humans", "Deployment for Humans", "requests: HTTP for Humans". He's even gone so far as to (facetiously?) trademark it ("args: Argument Parsing for Humans™"). Now everything with syntactic sugar is "for humans".

If you think your API represents a breakthrough in simplicity then trust me: it will do the talking for you.

Re:Schedule – Python job scheduling for humans by mmcclellan on Saturday October 03, 2015 @12:17PM (#10324367)
Note to self: don't describe strider (https://github.com/mpdehaan/strider) as Vagrant for humans.
Re:Schedule – Python job scheduling for humans by maxerickson on Saturday October 03, 2015 @09:28AM (#10323806)
The global scheduler object is provided for convenience, it is possible to create others.

I agree about the for humans branding.

I also don't find this api very appealing, but I haven't given much consideration to what an interval api should look like.

Re:Schedule – Python job scheduling for humans by rch on Saturday October 03, 2015 @10:40AM (#10324038)
It should probably look a lot like ISO 8601.