News for hackers. Stuff that matters.
Re:What happens if we remove 50 percent of Llama? by DHRicoF on Monday December 02, 2024 @11:27AM (#42297725)
I don't see that contuer-intuitive at all. If you have a barrier in your cost function in 1d model you have to cross over it no matter what. In 2d it could be only a mount that you can go around. More dimensions mean more ways to go around.
Re:What happens if we remove 50 percent of Llama? by fasa99 on Tuesday December 03, 2024 @03:40PM (#42311104)
This is also how the human brain works. A young babby will have something more similar to a fully connected network. Versus a Biden type elderly brain will be more of a sparse minimally connected feed forward net. The question is (1) can this be adjusted dynamically in silico and (2) if we succeed in that, does fine-tuning still work?
Re:What happens if we remove 50 percent of Llama? by scotty79 on Thursday December 05, 2024 @03:13PM (#42332215)
You don't have to compare to old age. Even 10 year old child has its brain pruned immensely when compared to its babyself.