News for hackers. Stuff that matters.
Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by zozbot234 on Wednesday March 27, 2024 @11:23AM (#39840494)
That's system memory, not unified memory. Unified means that all or most of it is going to be directly available to the Apple Silicon GPU.
Re:Introducing DBRX: A New State-of-the-Art Open LLM by Data... by giancarlostoro on Wednesday March 27, 2024 @02:43PM (#39843023)
This is the key factor here. I have a 3080, with 16GB of Memory, but still have to run some models on CPU since the memory is not unified at all.