JobRunr: A library for background processing in Java

Re:JobRunr: A library for background processing in Java by therealrootuser on Thursday May 25, 2023 @08:12PM (#36078140)

Seems like these job scheduling systems are a dime a dozen these days. Since we're an AWS shop, eventually my team ended up just building a system based on EventBridge and Fargate, killing off a previous system built on top of Quartz. Scheduling is all handled via Terraform. It's been solid for several years now, and costs next to nothing to operate. We can parallelize as much or as little as we want.

At the end of the day, I don't want more to run more dedicated boxes for yet another jobs systems. I just want to hand off a container to the ether and say "please run this container until it stops, and do this once an hour or once a day." I don't want to get alerts in the middle of the night telling me that the Quartz scheduler has had some esoteric failure, and I don't want Jobs A, B, and C to get killed because Job D started doing something dumb.

Having a nice UI is cool, but I would rather not have more servers and relational databases and Java-cron libraries that can do dumbness in the middle of the night.

[ Parent | on HN ]

Re:JobRunr: A library for background processing in Java by kernelbugs on Thursday May 25, 2023 @08:56PM (#36078473)

Any tips, tricks, or resources for getting started using Fargate for one-off or recurring jobs? I have terraform setup and managing AWS resources, but every time I look into Fargate it seems like guides point towards running webapps instead of diverse jobs.

[ Parent | on HN ]

Re:JobRunr: A library for background processing in Java by jamesfinlayson on Thursday May 25, 2023 @10:11PM (#36078928)

I had a quick look at something I've got (not written by me) and it looks like you create an EventBridge rule with a schedule expression and create an EventBridge target (which can include an ECS task: https://docs.aws.amazon.com/cli/latest/reference/events/put-...).

[ Parent | on HN ]

Re:JobRunr: A library for background processing in Java by inkyoto on Friday May 26, 2023 @12:58AM (#36080140)

You can use the aws_appautoscaling_scheduled_action terraform resource to create a scheduled scaling policy action to mimic a scheduled Fargate container fire-up, e.g. from zero Fargate container instances to one or however many are required, and then back down to zero.

[ Parent | on HN ]

Re:JobRunr: A library for background processing in Java by shpongled on Friday May 26, 2023 @01:46AM (#36080462)

I would look into AWS Batch - it works pretty well for running diverse jobs. I have a few jobs that are triggered by S3 uploads that run for 1-30 minutes, and other jobs that run for ~hours. All on Fargate

[ Parent | on HN ]

Re:JobRunr: A library for background processing in Java by sverhagen on Friday May 26, 2023 @12:16AM (#36079832)

Within Java, though, Quartz has ruled for years, it's aged, its website has been reorganized a few times so that half the search results end in dead links, it was time for a new contender. But my fear is that someone else takes the crown, with another business opportunity in mind, which is likely to fizzle too, and then the cycle just repeats. Another thread here was saying this is easy money, but are open source or open core or whatever companies really all that often a slam dunk?

[ Parent | on HN ]

Re:JobRunr: A library for background processing in Java by manigandham on Friday May 26, 2023 @07:15AM (#36082639)

> "are open source or open core or whatever companies really all that often a slam dunk?"

No, they're often not. Many struggle to ever make a profit.

[ Parent | on HN ]

Re:JobRunr: A library for background processing in Java by irl_chad on Friday May 26, 2023 @02:24AM (#36080725)

We came to the exact same conclusion. EventBridge time triggers a Fargate task. The job automatically terminates the process after execution, so the container shuts down and all is good.

[ Parent | on HN ]