Transformers Are Graph Neural Networks

Re:Transformers Are Graph Neural Networks by lexpar on Monday March 09, 2020 @11:04AM (#22525951)

Yeah, I guess this is fine if by "Transformers are Graph Neural Networks" we mean Transformers < GNN, rather than Tranformers == GNN.

"Sentences are a fully connected graph". Ok fine, but that's a graph with basically no information embedded in its structure. GNNs are supposed to be useful for graphs that have interesting structure, right?

[ Parent | on HN ]

Re:Transformers Are Graph Neural Networks by chaitjo on Tuesday March 17, 2020 @08:57PM (#22612485)

Not necessarily. In fact, applying the Transformer/GNN on a full graph is seen by some people as 'discovering' or identifying some useful underlying graph structure. We had an interesting discussion on this on Twitter: https://twitter.com/PetarV_93/status/1233820829852618754

[ Parent | on HN ]