Re:The dark side of Graph Neural Networks
by lmeyerov on Wednesday June 29, 2022 @07:17PM
(#31926924)
Yeps, that's a GCN, and probably the most popular GNN is indeed a RGCN :)
The transformer comment is interesting. They're very close, and in general, the tricks people use elsewhere are getting translated to GNNs: convolution, attention, ... . But scaling is still happening, so recent couple of years have gotten folks doing 1M-1B level, but not yet LLM scale yet. Critically, the scaling work is relatively recent -- good GPU impl, good samplers, etc -- and with a good trajectory.
We tracked GNNs for years but stayed away until heterogeneity + scaling started to get realistic for commercial workloads, and that's finally happening. Major credit to folks like deepmind, michael bronstein, early graphsage / jure, various individual researchers, and now aws+nvidia engineers for practical engineering evolutions here.
Re:The dark side of Graph Neural Networks
by MonkeyMalarky on Wednesday June 29, 2022 @07:28PM
(#31927005)
Coming from a position of knowing nothing about graph neural networks, this is the best description I've seen so far in this thread.
In that way, GNNs solve graph problems, the same way CNNs solve image processing problems. Training the GNN is more of an optimisation problem.
Edit: maybe graph theory can help with training on very large graphs but I don't know enough about that.