Re:Llama.cpp 30B runs with only 6GB of RAM now
by VernorVintage on Saturday April 01, 2023 @03:55PM
(#35403525)
It's a pretty big concern if you had to spend a billion on training, but 6 months later the open source community is able to replicate your training for <100K because you were too cheap to hire an extra 100 optimization experts
That'd be a 10,000 fold depreciation of an asset due to a preventable oversight. Ouchies.
That'd be a 10,000 fold depreciation of an asset due to a preventable oversight. Ouchies.