...That sounds more like an HR problem than a technical problem.

There are network engineers in the world who don't live in caves and gnaw at the bones of administrative assistants.

There are, but they are hard to find.

Networking is a great place for assholes to build empires and exert control. In 20 years of professional engagement with distributed and data center networks, most of these guys (and they are almost always guys) running network orgs are an impediment and spend more time helping out their vendor of choice than anything else.

I've run into awesome network guys in position of power only a few times, and they are 100x employees. The last major rollout of a service that I did was literally 3 months ahead of schedule solely because of the efforts of this awesome network dude snowman as recently put in charge.

We saw some good people, but they generally left because they didn't want to work in that environment.

Eventually replaced with someone helpful, it was amazing what we started to get done. Got lucky there we found the replacement and he stayed long enough to be there when needed.

I wonder if all these 10x network admins are now working at gcp/AWS/Azure.

Oh it was, but it was the situation we had to operate in.

And Amazon allowed to to achieve amazing things that never would have been possible and that's without the idea of things like Kinesis or Redshift that we couldn't have achieved easily/at all even without personnel problems. You simply could never get 3 new servers or drastically respec an existing one in under a day let alone minutes. We were a small shop so it's not like we had spare servers ready to go.

Please send me their resumes..

Is sabotage something you are worried about in general. Just seems like there might be deeper and serious issues if that is a primary​ concern.

It was sabotage in the 'oops this project isn't working' or 'oops this really import thing came up, no time' or 'didn't I do that?' sense. Having some plausible deniability. Work slowdown/stoppage stuff. Not actively breaking things (mostly). And it was all directed at maintaining control/power.

Worst person I've ever worked with. When I tell stories at future jobs I fully expect people to not believe me. Hell, people who weren't there for earlier incidents often didn't believer them.

Oh I believe you. I've heard of it before. Was trying to say that if that's an issue, it should be an HR fix more than an infrastructure fix. But I also imagine often that person also convinced management the are an "11x rockstar programmer"...

They were one of the hardest working people in the company. Partially that's because they worked on nonsense projects they made up, and partially that's because they were always putting out fires caused by the fact they did things poorly in the first place.

That's scares me cause I've seen that bit way too many times. There are huddled over the terminal in the evening as all managers leave "Oh look so and so is working so hard" and I am thinking "they are fixing the bug they put in just last week because they didn't bother testing it enough".

Tons of that. Each server was a literal copy of a previous one then modified to fit its new role. So we never got two servers that were the same. They'd have extra junk on them too.

Just basic docker use (or something similar) would have made a HUGE difference. Parts of the department were sent to training but it didn't happen because it would have made him less indispensable.

At one point someone higher up (but not high enough) asked how many servers we had in Amazon and the answer like 10x our physical set.

"And only one of you manages them?"

"On the side, we don't usually have to touch them."

"Then why does it take 4 people for the other servers?"

(Collectively) "Yeah. Why IS that?"

Change control was a big problem too. "Random" changes, things that didn't change at all (even when we could prove it), stuff like that causes SO MANY issues, basically all avoidable.