Re:The Cost Savings Of Netflix's Internal Spot Market
by naturalgradient on Tuesday December 05, 2017 @09:48AM
(#15851975)
So they built...a scheduler?
[ Parent
| on HN ]
Re:The Cost Savings Of Netflix's Internal Spot Market
by mnx on Tuesday December 05, 2017 @11:00AM
(#15852512)
Well, a multi-machine scheduler.
Re:The Cost Savings Of Netflix's Internal Spot Market
by robnagler on Tuesday December 05, 2017 @11:17AM
(#15852666)
In the HPC world, there are many of these: SLURM, Torque, PBS.
Re:The Cost Savings Of Netflix's Internal Spot Market
by toomuchtodo on Tuesday December 05, 2017 @12:16PM
(#15853279)
Don't forget Globus and Condor!
Re:The Cost Savings Of Netflix's Internal Spot Market
by pnutjam on Tuesday December 05, 2017 @04:37PM
(#15856261)
How could you overlook (son of) Grid Engine
Re:The Cost Savings Of Netflix's Internal Spot Market
by samstave on Tuesday December 05, 2017 @11:20AM
(#15852691)
Well yes, but the elegance here is how they layered their scheduler to be on top of scheduling the resources they paid for at AWS with a meta-scheduler, which when they originally did this was rather novel in the consumer market.
Re:The Cost Savings Of Netflix's Internal Spot Market
by brucephillips on Tuesday December 05, 2017 @06:23PM
(#15857118)
No, the underlying instances were reserved. It reduces to the same problem as scheduling on bare metal.
Re:The Cost Savings Of Netflix's Internal Spot Market
by samstave on Tuesday December 05, 2017 @09:17PM
(#15857982)
So, are you saying this is simplistic.?
What is your solution.?
Re:The Cost Savings Of Netflix's Internal Spot Market
by brucephillips on Wednesday December 06, 2017 @12:18AM
(#15858736)
It's not simplistic. It's simple. And a fine solution.
Re:The Cost Savings Of Netflix's Internal Spot Market
by samstave on Saturday December 09, 2017 @03:31PM
(#15887928)
Ah yes, good correction of the statement.
I think we both agree, just that I was more clumsy with my language.
Thanks
Re:The Cost Savings Of Netflix's Internal Spot Market
by svachalek on Tuesday December 05, 2017 @12:58PM
(#15853852)
Scheduling implies this is simply done on a clock, while this is more about real-time maximization of CPU resources with feedback based on measured load.
Re:The Cost Savings Of Netflix's Internal Spot Market
by hobofan on Tuesday December 05, 2017 @01:53PM
(#15854520)
> Scheduling implies this is simply done on a clock
That's a very outdated definition of scheduling in regards to cloud computing. The schedulers of Mesos, Omega, Borg, etc. all do allow for CPU-load based scheduling, and have schedulers for other scenarios.