Language models can explain neurons in language models

Re:Language models can explain neurons in language models by PaulHoule on Tuesday May 09, 2023 @05:58PM (#35880918)

I like the idea. Note that LLMs have some skill at decoding sequential dense vectors in the human brain

https://pub.towardsai.net/ais-mind-reading-revolution-how-gp...

so why not have them decode sequential dense vectors of their own activations?

As for the majority scoring poorly, they suggest that most neurons won't have clear activation semantics so that is intrinsic to the task and you'd have to move to "decoding the semantics of neurons that fire as a group"

[ Parent | on HN ]

Re:Language models can explain neurons in language models by Imnimo on Tuesday May 09, 2023 @11:43PM (#35883395)

I don't think this is showing LLMs performing decoding. They're just using the LLM to propose possible words. The decoding is done by using another model to score how well a proposed word matches brain activity, and using that score to select a most likely sequence given the proposals from the LLM.

[ Parent | on HN ]