I'm replying to that comment.

The issue is still the same: what do you do in the second year (or the third), after you've fixed all the low hanging fruit. If it takes a constant amount of time to investigate a particular service, it's not even worth examining the long tail, because the investigation is costlier than the savings. So once you've saved the high 8 figures by fixing the low hanging fruit in the biggest 20 services or whatever, what next?

The broader point is that while it is often very worthwhile to have individual employees work on optimizations, it is much less often worthwhile to task teams entirely with optimizations, especially in the way you describe. Having some group in charge of generalizable fleetwide optimizations is possibly useful. Doing some kind of resource quota thing where you get fewer resources than you're forecast to need (to force you to do some kind of local optimization) may make sense, but having a strike team whose job it is to tune JVM parameters isn't useful. A central team writing a document on best practices, setting better defaults, or using/building some kind of grid-search optimizer are all probably better investments.

The answer is you build a team that builds tools to make optimization easier for everyone in the engineering organization. One individual team can't optimize every application, but if you make it easy for every engineering team to profile their applications and debug performance problems, you've enabled every team to continuously optimize the low-hanging fruit. In effect, you're no longer solving one-off 8-figure problems, you're optimizing the time of other engineers which is something that pays off in perpetuity.

Good large engineering organizations have these teams, and understand their value. For smaller companies, it's not as clear if you can reasonably fund a team for that kind of work.

> The issue is still the same: what do you do in the second year (or the third), after you've fixed all the low hanging fruit.

Then you destaff the team, collect your tens of millions of dollars, and be happy about a successfully finished project? The problem with going too far down this line of reasoning is that you leave even low-hanging fruit laying around forever.

What you seem to be missing is that "destaffing" a team of a dozen people is really expensive and painful, and people don't generally sign on to jobs where we say "yeah you'll be doing this for two years and then you'll have to find a new role".

Like I've said twice now, you can solve this problem without stupid business practices, and the you're arguing with a straw man. If you have to say "going too far down this line", you're no longer responding to the argument presented, but a bad faith misrepresentation. That's inappropriate on HN.