I think the surprising thing is that it's the same exact neuron that does the different modes, not that the network has the ability to detect a photo of spiderman and a picture with the text "spiderman". You might instead imagine that the network would have some neurons specialized to reading text, and others specialized to recognizing faces, and those would be like two separate paths through the network.

I think the OP meant that the network writes spider-man in both cases. So the multi-modal "Spider-man" neuron is more a "write spider-man" neuron. It is impossible to tell (at the moment) if its forming and purpose is really comparable to how biological multi-modal "think about spider-man" neurons evolve and work. IMO it is not very probable.

Well, that's not really what's going on here. CLIP has two components - a text encoder that encoders candidate captions, and an image encoder. There's no part that does "writing" - it just makes an encoding for the image, and then sees which candidate text encoding is the most similar. Further, what's being looked at here, as I understand it, is JUST the image encoder part. The neuron in question isn't seeing or generating caption text, it's just a step along the way in trying to come up with a representation of the image. So it's surprising that that same neuron is strongly activated both by the word "spiderman" appearing in an image, and an actual picture of spiderman.

If the text encoder's representation of "spiderman" and "spider" are close together, then the image encoder is penalized if it makes the image representations far apart from each other. So what we are witnessing here could simply be the neuron that says "I think 'spider' appears in the caption'". Of course it could be that the network understands the metaphorical relationship of spiderman to spiders, but the simpler explanation seems more plausible to me.

Right, so I don't think the surprise is that the neuron responds to a picture of spiderman and a picture of a spider. What's surprising is that it responds to a picture of spiderman, and a picture of the text "spider". The thing that's interesting is not that it understands that the concept of a spider and the concept of spiderman are close together, it's that the same neuron is responsible for both the visual depiction of spiderman, and for an image of the text. In previous networks, you'd maybe have a neuron that detected a picture of text in general, but not one that would look for a particular word as well as the image it corresponds to.

If its deep in the network, though, couldn't you interpret it as a single neuron responsible for the substring "spider" in the caption? That seems less surprising. But, I see your point.

Yeah, I mean it's definitely the case that the signal here is that words occur in captions for pictures of text as for pictures of objects. But if you do this same experiment of visualizing dataset examples for other networks (like ones trained on Imagenet), you don't find this sort of multimodal neuron.

I am not sure if I get it, but isn't the CLIP "multi-modal" neuron just considered as "multi-modal" because it occurs some layers before the actual output? I am not sure, but maybe the indirection is just obfuscation and not a sign of abstraction.

I meant "write" not in a literal sense. "CLIP pre-trains an image encoder and a text encoder to predict which images were paired with which texts in our dataset" Isn't this the implicit coupling between text and image that is observed as multi-modal neurons?

Well, the text encoder sees the ascii characters s-p-i-d-e-r (after byte-pair encoding). That's different from seeing a photograph of a piece of paper that says "spider" on it. It's not surprising that the network can associate a picture of spiderman with a caption that contains the text "spider", but rather that the same neuron lights up when you show it a piece of paper that says "spider" as when you show it a picture of spiderman.

Maybe I don't get something about CLIP. But won't there the same labels and as a result the same pairings for a written piece of paper with spider on it and a picture of Spiderman?

The labels are just whatever people on the internet wrote next to the image. Certainly there are some instances of things like "this is a picture that says 'spider'" or whatever (probably a little more natural than that), or else the network would have no way of learning to read. But what's interesting here is that it's the same neuron doing the reading and doing the recognizing of Spiderman's head. That's not the only way that it could have solved the problem. There could have been some dimensions of the representation vector used for reading text, and other for recognizing visual objects, and those would be handled by separate subsets of neurons in the network.

Maybe it just recognizes pixel soups? Why should it know the difference between a piece of text and a real spider? It's just our interpretation of the image that makes it "multi-modal". CLIP probably just categorizes certain kinds of white and black patterns as a special kind of spider that happens to also look like a piece of paper and instances of text.

I mean, yeah, it does just recognize pixel soups - all the neurons are just semi-scrutable combinations of other features. It's probably the case that there are some early neurons that recognize various letters, and so you'd have some subset of neurons that are shared between the "spiderman" neuron circuit and the circuits that are used by other neurons that recognize other words. I don't know how much credit you'd give that for "reading", but I'd say it at least would qualify as multi-modal.

So if the network can recognize species by their rear view you would consider it also multi-modal? Because that is what I am trying to tell... there are probably no higher level concepts here, just various different pixel soups that happen to need the same label.

If the trained network produces common labels for sets of pixel soups that we consider to be semantically related, but are not visually related, then that is interesting.

I agree. But isn't it more probable that there forms some kind of arbitrary "OR" logic than "real abstraction" which is indicated by choosing the word "multi-modal".

I guess we see something like this:

  e.g. "Photo of spider" -> Hierarchy of pixel soups -> "Photo of spider" OR "Photo of word spider" OR "Spider rear view" OR "Spiderman" OR ... -> [Spider]

What I think the authors want to tell me when calling it multi-modal:

  "Photo of spider" -> "Characteristics of a spider" -> [Spider]
  "Photo of word spider" -> "Letters S-P-I-D-E-R" -> "Written word spider" -> [Spider]

I guess there are two ways of looking at that question.

One is just basic generalisation - do these neurons effectively capture things within their semantic group (whatever that means) but completely outside of the training data. If yes, then I guess the answer might be 'yes, in some sense it is like a "real abstraction"'.

Second, and (afaik) currently a more philosophical framing - it isn't obvious whether "(sufficiently advanced) OR logic" and "real abstraction", are actually different. Additionally, for the purposes of a model like this one, I find it hard to see how they could be different. The best the model can do is (roughly speaking) assign neurons to particular concepts, be they ones that fit with our mental models of the world, or ones that are more "functional". The better a job it can do of the former, the more we might be inclined to believe that it is modelling things as we understand them.

A long time ago I implemented a Neocognitron. As far as I understand most deep learning models are very similar in architecture. My Neocognitron could recognize 1 and 0 (grey scale). I handcrafted the weights and hierarchy (as you do it with a normal beginner Neocognitron). When you wrote a 1 into the hole of the 0 or multiple 0s next to a 1, you could see how it weighted sometimes the 1 or the 0 higher. It's not very far fetched to add an "OR" layer that can combine patterns of 1 and 0. Yet I wouldn't call this layer multi-modal. I guess in case of CLIP due to the training method and the output alignment that this kind of OR layer simply occurs in some cases.

When I talked about OR logic I meant a simple and direct aggregation of all possible pixel soups that should be labeled in a specific way. Calling this multimodal and comparing it to the Jennifer Aniston neuron is framing the situation in a way that is not good for science (see AI winter). Especially when multimodal neurons in neuroscience refer to processing more than one sense.

I find it hard to imagine a different solution than sufficiently advanced OR logic for real abstraction within the current ANN models. That does not mean there is none within ANNs and especially not in the brain. We are far from being able to do anything like the brain does and don't know how it learns new concepts so fast - without thousands of sample images. Just because we observe an effect that reminds us of some vague findings in the brain and we call the effect the same in ANNs does not make ANNs more like brains.

BTW nice podcast that also discusses the Jennifer Aniston neuron with its "discoverer": https://soundcloud.com/neuropodcast/episode62

The basic idea of having concept aggregation to match incoming and outgoing information in various modes is applicable to both.