Re:Scaling Monosemanticity: Extracting Interpretable Feature...
by rcxdude on Wednesday May 22, 2024 @08:55PM
(#40449058)
I mean, it's mostly about how close concepts are to each other, and to some extent how different concepts are placed on a given axis. Of course the concept space is very high-dimensional so it's not very easy to visualise without reducing the dimensions, but because we mostly care about distance that reduction is not particularly lossy, but it does mean that top-left vs bottom right doesn't mean much, it's more that mcdonalds is usually closer to food than it is to, say, gearboxes (and that a representation that doesn't do that probably doesn't understand the concepts very well).
[ Parent
| on HN ]
There are no comments on this comment yet.