Re:Ollama now supports AMD graphics cards
by eclectic29 on Friday March 15, 2024 @07:50PM
(#39721907)
Of course, you can support concurrent requests. But Ollama doesn't support it and it's not meant for this purpose and that's perfectly ok. That's not the point though. For fast/perf scenarios, you're better off with vllm.
[ Parent
| on HN ]