Do large language models need all those layers?

Re:Do large language models need all those layers? by fluidcruft on Friday December 15, 2023 @05:38PM (#38659702)

I was a RSNA this year (major Radiology conference and trade show) and one of the presenters made the claim that their model was generalized because it works on different body parts. My intuition was they were claiming they trained it on one body parts and it subsequently worked on different body parts (which could be convincing). So at first this seemed fine. But in reality they had trained the same model on all of those body parts. That really go me to thinking about the old myth that we only use 10% of our brains. Anyway I think capacity would be when the model can no longer learn.

But anyway it made me wonder if there's a way to measure "what x% of a model is actually used" similar to the myths about human brains.

[ Parent | on HN ]

Re:Do large language models need all those layers? by cyanydeez on Friday December 15, 2023 @06:19PM (#38660093)

"junk" DNA is a similarly dark data

[ Parent | on HN ]

Re:Do large language models need all those layers? by jakobnissen on Saturday December 16, 2023 @06:17AM (#38663438)

An often repeated claim, but untrue. We have a pretty good understanding of large parts of junk DNA, how it got there and what it is, and know that most of it really is junk.

That being said, only around 2% of the human genome is coding, with perhaps another one or two percent having known no coding function. And estimates of functional DNA is somewhere around 8% of the genome. So, most functional DNA is still unknown.

[ Parent | on HN ]

Re:Do large language models need all those layers? by cicce19 on Friday December 15, 2023 @10:48PM (#38661545)

I was also at RSNA. What was the model intended to do in this example?

[ Parent | on HN ]

Re:Do large language models need all those layers? by fluidcruft on Saturday December 16, 2023 @09:01AM (#38664242)

IIRC it was about emerging MRI reconstructions for MSK in one of the educational sessions. It was more the ambiguous way the claim was stated about trasferrable knowledge initially that got me thinking about the "10% of brain" thing rather than anything specifically presented.

[ Parent | on HN ]