Re:Mistral CEO confirms 'leak' of new open source AI model n...
by moffkalast on Thursday February 01, 2024 @04:18AM
(#39214092)
In production you'd need enough GPUs to load the entire thing so it serves fast enough, so it makes sense to go with AWQ or EXL2 instead since they're optimized for that, but I think they were sending it out for people to just test and evaluate, probably on less capable machines, in which case GGUF is king.
[ Parent
| on HN ]
There are no comments on this comment yet.