News for hackers. Stuff that matters.
Re:Llama.cpp 30B runs with only 6GB of RAM now by catchnear4321 on Friday March 31, 2023 @07:26PM (#35395165)
Money did.

Why waste developer hours (meaning effort) if you can just scale the infra for a little cash? Do it in small enough increments and the increases only outweigh FTEs if you consider all scaling events and look at a long enough time scale.

Suddenly it takes way too much for way too little, but it cost half as many overpaid developers who can’t be arsed to performance.

Edit: in case that sounds like the opposite of intended, ggerganov and jart are the outliers, the exception.

There are no comments on this comment yet.