Re:Ollama is now available on Windows in preview
by jart on Saturday February 17, 2024 @04:15PM
(#39413621)
Zen4 AVX512 must be really good then.
Re:Ollama is now available on Windows in preview
by imtringued on Sunday February 18, 2024 @05:38AM
(#39417963)
To be fair a lot of the GPU edge comes from fast memory. A GPU with 20tflops running a 30 billion parameter model has a compute budget of 700flops per parameter. Meanwhile the sheer size of the model prevents you from loading it more than 20 times from memory per second.
This is one of the worst acts of self-sabotage I have ever seen in the tech business.