News for hackers. Stuff that matters.
Re:Ollama is now available on Windows in preview by chown on Saturday February 17, 2024 @03:05PM (#39412979)
As others have mentioned, Ollama uses Llama.CPP under the hood and they recently released Vulkan support which is supposed to work with AMD GPUs. I was able to use llama.cpu compiled with Vulkan support with my app [1] and make it run on an AMD laptop but I was unable to make it work with Ollama as it makes some assumptions about how it goes about searching for available GPUs on a machine.

[1]: https://msty.app

Re:Ollama is now available on Windows in preview by Kelteseth on Saturday February 17, 2024 @04:31PM (#39413731)
I got a Windows defender Virus alert after executing your app.
Re:Ollama is now available on Windows in preview by chown on Saturday February 17, 2024 @06:23PM (#39414522)
Ugh! Probably because it’s an exe app? Not sure how to go around about that. I am looking into getting it signed just like the counterpart MacOS app. Thank you for the heads up and sorry about the false positive.
Re:Ollama is now available on Windows in preview by rezonant on Saturday February 17, 2024 @11:34PM (#39416241)
Ironically Ollama is also struggling with this sort of thing, see https://github.com/ollama/ollama/issues/2519

Code signing helps by having an avenue by which you can establish reliable reputation, and then using VirusTotal to check for AV flags and using the AV vendor's whitelist request form is the second part, over time your reputation increases and you don't get flagged as malware.

It seems to be much more likely with AI stuff, apparently due to use of CUDA or something (/shrug)

Re:Ollama is now available on Windows in preview by 65a on Saturday February 17, 2024 @04:54PM (#39413913)
ROCm is preferred over vulkan for AMD GPUs, performance wise. Using OpenCL or Vulkan should only be for older cards or weird setups.
Re:Ollama is now available on Windows in preview by chown on Saturday February 17, 2024 @06:24PM (#39414523)
That’s good to know. Thank you!