News for hackers. Stuff that matters.

A comment on Do vision transformers see like convolutional neural networks?. Parent comment

Re:Do vision transformers see like convolutional neural netw... by abalaji on Wednesday August 25, 2021 @12:28PM (#28303634)
yeah there has definitely been work done in that space: it’s called multi-modal models

not sure if this is the latest work but here’s some results from Google’s AI Blog

https://ai.googleblog.com/2017/06/multimodel-multi-task-mach...

[ Parent | on HN ]
Re:Do vision transformers see like convolutional neural netw... by im3w1l on Wednesday August 25, 2021 @02:49PM (#28305393)
What would be really cool is neural networks with routing. Like circuit switching or packet switching. No idea how you would train such a beast though.

Like imagine the vision part making a phonecall to the natural language part to ask it for help with something.

[ Parent | on HN ]
Re:Do vision transformers see like convolutional neural netw... by TylerLives on Wednesday August 25, 2021 @03:35PM (#28305904)
Sounds like The Society of Mind - https://en.m.wikipedia.org/wiki/Society_of_Mind
[ Parent | on HN ]
Re:Do vision transformers see like convolutional neural netw... by wittenator on Wednesday August 25, 2021 @04:57PM (#28306886)
Capsule networks have a routing algorithm as far as I know
[ Parent | on HN ]