News for hackers. Stuff that matters.
Re:Nvidia DGX GH200: The First 100 Terabyte GPU Memory System by Tepix on Wednesday May 31, 2023 @03:42AM (#36135608)
> LLaMA can apparently run quantized to 4 bits per param (not sure if worth it though)

From the GPTQ paper https://arxiv.org/abs/2210.17323:

"... with negligible accuracy degradation relative to the uncompressed baseline"

There are no comments on this comment yet.