News for hackers. Stuff that matters.
Re:A network engineer in search of greener pastures by alephnerd on Thursday August 01, 2024 @03:05PM (#41132432)
It's still additional operational overhead for marginal RoI.

If you're an organization that already has a SRE or Infra Eng hired, there's no point supporting a niche setup when every other team is using a prebuilt and troubleshooted environment.

Edit: Can't reply to OP atm, but he makes a good point. For a higher margins and less commodified segment like ML Compute, on-prem computing still makes financial sense.

That said, that's still managed by actual SWEs not a generic Sysadmin, due to intricacies of GPU Architecture and Model Development, and OP is extremely underskilled for this role.

Re:A network engineer in search of greener pastures by shortrounddev2 on Thursday August 01, 2024 @03:08PM (#41132464)
I think if all you're running is web microservices, then it's probably not worth it. But my company builds machine learning models offline and I feel that if we bought a dedicated GPU machine (literally just build one and put it in the office, not a rack mounted thing), we would save money in the long run after building a custom training pipeline