News for hackers. Stuff that matters.
Re:Streaming: a skill gap? by Jach on Monday February 03, 2020 @03:51PM (#22228916)
> In terms of business value, a cron job running in a high memory vps will more than satisfy and take much less time to develop.

Yeah, but I hate it... I've worked with a team where we had a cron job do some batch processing every night, but for some large customers it started taking ~12-15 hours to complete, and certain important user operations are locked while it's running. The solution? Running once per week starting on the weekend, with a manual trigger for customers who really need the results ASAP. Tiny effort for that easy fix and the team can continue working on new features, all-sized customers are still mostly happy, but Dijkstra would not have liked this...

Re:Streaming: a skill gap? by cultofmetatron on Friday February 07, 2020 @04:56AM (#22264551)
I would say this is the 5% of the time where it suddenly becomes worth it to streamify. you'll already have code showing what transformations need to be done. The point is to avoid premature optimization.
Re:Streaming: a skill gap? by hilbertseries on Monday February 03, 2020 @09:11PM (#22231686)
How long until the cron start takes 70 hours to finish?