News for hackers. Stuff that matters.
Re:Run llama3 locally with 1M token context by nicognaw on Wednesday May 01, 2024 @01:07AM (#40219708)
If you got random tokens on Apple Silicon Macs, you should check your stderr for Metal’s error code, like 5 for out of metal memory. Metal lets you use half of your unified memory as metal memory.

There are no comments on this comment yet.