News for hackers. Stuff that matters.
Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by c1b on Wednesday March 27, 2024 @06:08PM (#39845214)
> The difference is that the A6000 absolutely smokes the Mac.

Memory Bandwidth : Mac Studio wins (about the same @ ~800)

VRAM : Mac Studio wins (4x more)

TFLOPs: A6000 wins (32 vs 38)

Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by bevekspldnw on Wednesday March 27, 2024 @06:23PM (#39845326)
VRAM in excess of the model one is using isn’t useful per se. My use cases require high throughput, and on many tasks the A6000 executes inference at 2x speed.