Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data...
by Zambyte on Friday March 29, 2024 @12:14AM
(#39860626)
Nice, that's definitely a sweet deal
Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data...
by XCSme on Friday March 29, 2024 @12:23AM
(#39860680)
Thanks, I chose a 3090 instead of 4070ti, it was around $200 cheaper and has 24GB vs 16GB VRAM and a similar performance. The only drawback is the 350W TDP.
I still struggle with the RAM issue on Ollama, where it uses 128GB/128GB RAM for Mixtral 24.6GB, even though Docker limit is set to 90GB.
Docker seems pretty buggy on Windows...
Funnily, I think the card is new (smells new) and unused, most likely a scalper bought it and couldn't sell it.