News for hackers. Stuff that matters.
Re:LLaMA Now Goes Faster on CPUs by samus on Monday April 01, 2024 @03:27AM (#39891642)
Are they though? They are lossy compressing trillions of tokens into a few dozen GB. The decompression action is fuzzy and inefficient though.
Re:LLaMA Now Goes Faster on CPUs by raincole on Monday April 01, 2024 @04:16AM (#39891846)
And it requires massive computational power to decompress, which I don't expect to be available in a catastrophic situation where humans have lost a large chunk of important knowledge.
Re:LLaMA Now Goes Faster on CPUs by samus on Monday April 01, 2024 @04:39AM (#39891945)
I don't necessarily agree. It requires massive computing power, but running models smaller than 70G parameters is possible on consumer hardware, albeit slowly.
Re:LLaMA Now Goes Faster on CPUs by threecheese on Monday April 01, 2024 @06:31PM (#39900176)
Parent may be thinking more along the lines of a “hope we can print all the knowledge“ type catastrophe. Though if there is zero compute it’ll be tough reading all those disks!