News for hackers. Stuff that matters.
Re:Language models can explain neurons in language models by m1el on Tuesday May 09, 2023 @02:26PM (#35878467)
You're correct to have a suspicion here. Hypothetically the explainer could omit a neuron or give a wrong explanation for the role of a neuron. Imagine you're trying to understand a neural network, and you spend enormous amount of time generating hypotheses and validating them. Well the explainer might give you 90% correct hypotheses, it means you have 10 times less work to produce hypotheses. So if you have a solid way of testing an explanation, even if the explainer is evil, it's still useful.

There are no comments on this comment yet.