Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data...
by bevekspldnw on Wednesday March 27, 2024 @06:23PM
(#39845326)
VRAM in excess of the model one is using isn’t useful per se. My use cases require high throughput, and on many tasks the A6000 executes inference at 2x speed.
Memory Bandwidth : Mac Studio wins (about the same @ ~800)
VRAM : Mac Studio wins (4x more)
TFLOPs: A6000 wins (32 vs 38)