Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel
by Spooky23 on Wednesday October 11, 2023 @07:48PM
(#37851844)
Depends on the financials.
My employer has generated its own electricity and steam for decades.
For a small business - different story.
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel
by ClimaxGravely on Wednesday October 11, 2023 @10:08PM
(#37852697)
I hate to say this but it has gotten to the point where I'm starting to farm some of my own food (I'm starting to get fed up with produce quality issues in my hometown).
Still haven't started on the silverware or shoes yet.
I do agree with you though. If you are a non-tech company or a company that lacks the human resources you might as well go with the cloud.
If I wanted to host a website, sure, I can build a server out of parts and negotiate with my ISP and get a business pipe and handle all caching and such. Or like I can pay a provider $5/mo and get better performance and reliability with no management overhead. Yeah, maybe over 5 years I'd save more money doing it myself... but it's not worth the time.
If I wanted to generate a photo or a dozen, or a few paragraphs of text, that's like a few cents worth of cloud AI. Maybe low single-digit dollars. Or I could spend thousands on fat GPUs or a Macbook, spend forever training it, and still end up with a sub-par result.
AWS is profitable not just because they're overcharging you but because they are providing a hugely useful service for millions of businesses that don't want to deal with that infrastructure themselves, any more than they'd want to manage their own plumbing or electrical grid or roads and bridges leading to their office. DIY makes sense if you're doing it as a hobby or if your scale is so big that you would incur significant savings to in-house it, but for millions of small and medium businesses, it's just not the most practical approach. Nothing wrong with that.
I mean, it's like saying development is such a lost art... why hire a dev if you can learn to code yourself? Sure, but not everyone wants to, can, or has time.