inowland

Thats a nice reference, but those slides agree with me. He is explicitly talking about "ICMP Generation" getting punted, exactly as I am.

While ICMP isn't as reliable, I think the reasons are a little different. Its not that ICMP gets punted to the management plane in general - i.e. when you "ICMP echo request" ping a host, that packet doesn't go through the management plane of all the network devices in between. You know that as ping doesn't have the latency/loss problems traceroute does.

Rather what is going on is most network device data planes are going to punt _all_ IP packet's "time-to-live exceeded" case to the management plane, and so it will have the problems you describe (latency, throttling).

So why is ICMP still worse? First of all it misses having the UDP/TCP "port" information that allows it to be flow hashed through different routes. Secondly if the route hits any firewalls, they often have very different configuration for ICMP then they do for the TCP/UDP that they are configured to let through.

A half answer here is, given "agile devops" software development (i.e. developing with iterative tradeoffs on both features and operational stability), then regular meeting time matters a lot to resolve conflict, and so working time zone matters a lot. The more spread, and the more you get people blaming "not good enough processes" on what is really just "not enough common time to meet to collaborate".

Now the other half of that is what remote-first, "all I need is more time to code" developers really don't want to hear, is that in resolving conflict, in-person communication is much higher bandwidth than even zoom communication, both for the whiteboard, and for the fact being in-person de-escalates emotion. This is particularly true for early career developers, where some pushback (i.e. conflict) in their naive assumptions is actually how they are going to learn better from senior developers.

So while 5 days a week in the office is dead, its way too early to say what the eventual outcome will be. But I do agree less days a week will force some amount of square-foot downsizing, so I am glad I don't own any office buildings.

So not to totally contradict, but when this was happening, I was at AWS in management in its largest revenue business (EC2), and while cost was never an afterthought, management worked hard to prevent most engineers from having to be overly concerned with this vs the other major engineering tradeoff factors (features, reliability, engineering time). i.e. we would give constraints at the "cost" level, but they were abstract, more in terms of moving forwards from existing solutions.

In particular, there was a strong understanding that engineers would overfocus on optimizing for what was asked, and so the deliberate overfocus was on "thinking big" on products that could grow rapidly while having pricing and engineering models that may be neutral margin in the short term; only in the long term paying off as positive margin (without needing to "surprise" customers with a price change).