Re:Language models can explain neurons in language models
by m1el on Tuesday May 09, 2023 @02:26PM
(#35878467)
You're correct to have a suspicion here. Hypothetically the explainer could omit a neuron or give a wrong explanation for the role of a neuron.
Imagine you're trying to understand a neural network, and you spend enormous amount of time generating hypotheses and validating them.
Well the explainer might give you 90% correct hypotheses, it means you have 10 times less work to produce hypotheses.
So if you have a solid way of testing an explanation, even if the explainer is evil, it's still useful.
[ Parent
| on HN ]
There are no comments on this comment yet.