Llama 3.1 in C - Hackdot

Re:Llama 3.1 in C by impossiblefork on Wednesday July 24, 2024 @04:55AM (#41054982)

There's actually an old paper titled Optimal Brain Damage, where they don't try to find optimal quantizations, but optimal sparse versions of a models-- i.e. where some weights are set to zero.

[ Parent | on HN ]

Re:Llama 3.1 in C by AMICABoard on Wednesday July 24, 2024 @06:28AM (#41055503)

Interesting, I heard something like that, but now I must read about it.

[ Parent | on HN ]

Re:Llama 3.1 in C by nickpsecurity on Wednesday July 24, 2024 @10:06AM (#41057264)

I found…

Optimal Brain Damage https://www.researchgate.net/publication/221618539_Optimal_B...

Optimal Brain Compression https://openreview.net/pdf?id=ksVGCOlOEba#:~:text=The%20resu....

TinyVolt’s implementation of it: https://github.com/TinyVolt/optimal-brain-compression

[ Parent | on HN ]

Re:Llama 3.1 in C by AMICABoard on Wednesday July 24, 2024 @10:27AM (#41057459)

Thanks, I'll read it up. Interesting.

[ Parent | on HN ]

Re:Llama 3.1 in C by nickpsecurity on Wednesday July 24, 2024 @10:35AM (#41057537)

That was really for them. You’re out there building neat stuff. Your talent might warrant looking into AdderNets and Bitnets which might get the cost down. There’s also some brain-inspired designs.

I don’t think many people have implemented such things. You might discover something new experimenting with them.

[ Parent | on HN ]

Re:Llama 3.1 in C by AMICABoard on Wednesday July 24, 2024 @12:35PM (#41058815)

[ Parent | on HN ]