News for hackers. Stuff that matters.

A comment on Serving AI from the Basement – 192GB of VRAM Setup. Parent comment

Re:Serving AI from the Basement – 192GB of VRAM Setup by nrp on Sunday September 08, 2024 @02:25PM (#41482096)
How are you finding 2b/3b quantized llama 405B? Is it behaving better than 8b or 16b llama 70B?
[ Parent | on HN ]

There are no comments on this comment yet.