News for hackers. Stuff that matters.
Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by kurtbuilds on Wednesday March 27, 2024 @09:26AM (#39838751)
What’s the process to deliver and test a quantized version of this model?

This model is 264GB, so can only be deployed in server settings.

Quantized mixtral at 24G is just small enough where it can be running on premium consumer hardware (ie 64GB RAM)

There are no comments on this comment yet.