Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Introducing DBRX: A New State-of-the-Art Open LLM by Databricks.
This model is 264GB, so can only be deployed in server settings.
Quantized mixtral at 24G is just small enough where it can be running on premium consumer hardware (ie 64GB RAM)
There are no comments on this comment yet.
This model is 264GB, so can only be deployed in server settings.
Quantized mixtral at 24G is just small enough where it can be running on premium consumer hardware (ie 64GB RAM)