Re:LLaMA Now Goes Faster on CPUs
by raincole on Monday April 01, 2024 @01:53AM
(#39891250)
It seems to be an unbelievably inefficient way to back up knowledge.
[ Parent
| on HN ]
Are they though? They are lossy compressing trillions of tokens into a few dozen GB. The decompression action is fuzzy and inefficient though.
And it requires massive computational power to decompress, which I don't expect to be available in a catastrophic situation where humans have lost a large chunk of important knowledge.
I don't necessarily agree. It requires massive computing power, but running models smaller than 70G parameters is possible on consumer hardware, albeit slowly.
Re:LLaMA Now Goes Faster on CPUs
by threecheese on Monday April 01, 2024 @06:31PM
(#39900176)
Parent may be thinking more along the lines of a “hope we can print all the knowledge“ type catastrophe. Though if there is zero compute it’ll be tough reading all those disks!