Re:Bringing K/V context quantisation to Ollama
by magicalhippo on Thursday December 05, 2024 @03:47AM
(#42326237)
Aa a Windows user, who just wanted something bare bones for playing, I found this[1] small project useful. It does support multi-modal models which is nice.
[1]: https://github.com/jakobhoeg/nextjs-ollama-llm-ui