News for hackers. Stuff that matters.
Re:The Cost Savings Of Netflix's Internal Spot Market by naturalgradient on Tuesday December 05, 2017 @09:48AM (#15851975)
So they built...a scheduler?
Re:The Cost Savings Of Netflix's Internal Spot Market by mnx on Tuesday December 05, 2017 @11:00AM (#15852512)
Well, a multi-machine scheduler.
Re:The Cost Savings Of Netflix's Internal Spot Market by robnagler on Tuesday December 05, 2017 @11:17AM (#15852666)
In the HPC world, there are many of these: SLURM, Torque, PBS.
Re:The Cost Savings Of Netflix's Internal Spot Market by toomuchtodo on Tuesday December 05, 2017 @12:16PM (#15853279)
Don't forget Globus and Condor!
Re:The Cost Savings Of Netflix's Internal Spot Market by pnutjam on Tuesday December 05, 2017 @04:37PM (#15856261)
How could you overlook (son of) Grid Engine
Re:The Cost Savings Of Netflix's Internal Spot Market by samstave on Tuesday December 05, 2017 @11:20AM (#15852691)
Well yes, but the elegance here is how they layered their scheduler to be on top of scheduling the resources they paid for at AWS with a meta-scheduler, which when they originally did this was rather novel in the consumer market.
Re:The Cost Savings Of Netflix's Internal Spot Market by brucephillips on Tuesday December 05, 2017 @06:23PM (#15857118)
No, the underlying instances were reserved. It reduces to the same problem as scheduling on bare metal.
Re:The Cost Savings Of Netflix's Internal Spot Market by samstave on Tuesday December 05, 2017 @09:17PM (#15857982)
So, are you saying this is simplistic.?

What is your solution.?

Re:The Cost Savings Of Netflix's Internal Spot Market by brucephillips on Wednesday December 06, 2017 @12:18AM (#15858736)
It's not simplistic. It's simple. And a fine solution.
Re:The Cost Savings Of Netflix's Internal Spot Market by samstave on Saturday December 09, 2017 @03:31PM (#15887928)
Ah yes, good correction of the statement.

I think we both agree, just that I was more clumsy with my language.

Thanks

Re:The Cost Savings Of Netflix's Internal Spot Market by svachalek on Tuesday December 05, 2017 @12:58PM (#15853852)
Scheduling implies this is simply done on a clock, while this is more about real-time maximization of CPU resources with feedback based on measured load.
Re:The Cost Savings Of Netflix's Internal Spot Market by hobofan on Tuesday December 05, 2017 @01:53PM (#15854520)
> Scheduling implies this is simply done on a clock

That's a very outdated definition of scheduling in regards to cloud computing. The schedulers of Mesos, Omega, Borg, etc. all do allow for CPU-load based scheduling, and have schedulers for other scenarios.