Also, locality and latency can both be expressed in terms of placement rules and schedulers on Mesos can use those rules to guarantee or express preference for task placement that optimizes around reduced latencies.
The point that I take away is that the ability to express your needs in a declarative way (e.g., "place these two tasks such that they have such-and-such latency) is much more scalable, flexible and resilient than coding to machine-specific internals. The latter is easier to update and supports delegation of responsibilities.
John Wilkes of Google puts it this way:
"Our own experience has been that allowing our developers unfettered access to the internals of infrastructure systems has been a problem, and we're moving away from that model as fast as we can.
Constructing large-scale complex systems with many interdependencies leads to brittle, fragile systems if they rely on internal implementation mechanisms.
Allowing internal customers to rely on internal implementation mechanisms has made it hard to adopt new technologies, because we only know what knobs they set - not why.
The fix for both is similar: describe the desired end state, not how to get there."
That makes total sense to me, thanks.
It seems like what might be ideal would be a specification language that doesn't care where things are, an implementation that tries to deal with that automagically, and a way to specify portions (to all) precisely that is checked against the high-level specification.
With the direction things are moving, "the data center as a computer" is absolutely the right approach.
Surely you meant 10's of milliseconds. But even so, fastest SSD random access latencies are in sub-millisecond ranges.
> Exposing machines as the abstraction to developers unnecessarily complicates the engineering, causing developers to build software constrained by machine-specific characteristics, like IP addresses and local storage.
It reminded me of the old RPC approach of making potentially any method call a remote method call. It didn't work out, simply because remote call latencies are orders of magnitudes higher than for in-process calls, so you need different (coarser) APIs for them.
By the same token, while abstractions are very welcome, they should still allow distinctions between local and non-local resources, for various definitions of "local".