shsshs

Ding, ding, ding. It’s the connections between the symbols which is what the transformer works with. The fact those symbols are “words” is uninteresting IMO. It works on words because these have connections in language. It works with anything that has a non-random influence on each other.

Peanuts aren’t nuts.. I know. We’re doing this.

So guys, we are really doing this?

We take 99% of what makes an LLM tick, claim we don’t use “words” and then say we found a completely novel new method and why didn’t anyone think of this?

Words are just symbols. We interpret it as language. The distinction is meaningless IMO.

He uses the mechanism that makes LLMs clever but he doesn’t use “words” but some other symbols. Transformers can do images and videos and whatever, they are not tied to to language.