News for hackers. Stuff that matters.
Re:Ollama now supports AMD graphics cards by eclectic29 on Friday March 15, 2024 @07:50PM (#39721907)
Of course, you can support concurrent requests. But Ollama doesn't support it and it's not meant for this purpose and that's perfectly ok. That's not the point though. For fast/perf scenarios, you're better off with vllm.
Re:Ollama now supports AMD graphics cards by airocker on Friday March 15, 2024 @07:53PM (#39721937)
Thanks! This is great to know.