Re:Mamba Explained
by moffkalast on Sunday March 31, 2024 @06:11AM
(#39882925)
Well consumer hardware can run something in the order of ~50B quantized at a "reasonable" price today, we'd need about 5 or 6 doublings to run something that would be GPT 4 tier at 1T+. So, it would need to continue for roughly a decade at least?
Current models are horrendously inefficient though, so with architectural improvements we'll have something of that capability far sooner on weaker hardware.
Current models are horrendously inefficient though, so with architectural improvements we'll have something of that capability far sooner on weaker hardware.