News for hackers. Stuff that matters.

A comment on Transformers Are Graph Neural Networks.

Re:Transformers Are Graph Neural Networks by treaheraher on Monday March 09, 2020 @07:50AM (#22524631)
Sorry but I hate it. Graph->graph transform can describe almost any operation, it's a superset of linear transforms. When all you know about your model is that 'it's a graph', you barely know anything at all. And transformers are just glorified AST parsers.
[ Parent | on HN ]
Re:Transformers Are Graph Neural Networks by cochne on Tuesday March 10, 2020 @09:34AM (#22535297)
> And transformers are just glorified AST parsers.

Can you explain this?

[ Parent | on HN ]
Re:Transformers Are Graph Neural Networks by chaitjo on Tuesday March 17, 2020 @08:45PM (#22612412)
I am guessing he means that Transformers 'discover' parse trees in sentences, as they operate on fully-connected graphs but end up focusing on the most important connections through attention.
[ Parent | on HN ]