Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel
by sbarre on Thursday October 12, 2023 @11:18AM
(#37858424)
Sure, like any other high-level fictional situation you can surely come up with many valid fictional counter-points of your own, but cloud hosting is popular for a reason.
And I think in most cases companies want to focus their employees and efforts on their core business, and if that doesn't include setting up and maintaining hardware in the long-term, then you don't build, you buy.
And I think in most cases companies want to focus their employees and efforts on their core business, and if that doesn't include setting up and maintaining hardware in the long-term, then you don't build, you buy.