Re:Deep Graph Library: Easy Deep Learning on Graphs
by brokensegue on Tuesday December 22, 2020 @07:44PM
(#25513189)
Anyone know how this compares to Facebook's recentish work on PyTorch-BigGraph (https://ai.facebook.com/blog/open-sourcing-pytorch-biggraph-...). They claim to be able to embed billions of nodes. I presume this library isn't distributed (quick glance suggests not).
Re:Deep Graph Library: Easy Deep Learning on Graphs
by 101101001010 on Tuesday December 22, 2020 @09:08PM
(#25513646)
They serve different purposes. PyTorch-BigGraph (BG) implements a set of algorithms for learning node embeddings (vector representations of each node in the graph) based on the edges (relations) present on a single graph.
dgl is a library for graph neural networks (GNNs). The algorithms present in BG can be implemented in dgl, albeit much less efficiently but the reverse might not be true.
More specifically GNNs are a set of methods based on what is called "Message Passing" algorithm, where the embedding of each node is a function (parameterized over the model weights) of its neighborhood and the edges that connect the node to it.
Additionally GNNs target learning functions that work on multiple graphs for example graphs of molecules to predict their properties, not just a single graph.
dgl can be compared to PyTorch Geometric. The former works on both TF 2.0 and Pytorch while the latter is only for PyTorch.
Both are almost equivalent, although dgl has some institutions backing it. PyTorch Geometric might feel a bit more lightweight to integrate in existing codebases.
Re:Deep Graph Library: Easy Deep Learning on Graphs
by sooheon on Wednesday December 23, 2020 @07:58AM
(#25517113)
The latter. It does not infer graphs from unstructured data, it gives a set of backend agnostic APIs for constructing featurized graphs, message passing and pooling/aggregating (with gradients being handled by your framework of choice), and has some really nice helpers like performing softmax over certain node/edge types, while keeping everything performant.
Re:Deep Graph Library: Easy Deep Learning on Graphs
by visarga on Wednesday December 23, 2020 @01:01AM
(#25514861)
It's a generalization of a fully connected layer, where you give the network an adjacency matrix for data routing between nodes. This allows for designing sparse data flows and learning relations between pairs of input elements.
Graph NN layer: f(X, A) = g(AXW), where g is an activation
Fully connected layer: f(X) = g(XW), like GNN where the adj. matrix A=I
To put it in context a few years ago GNNs became a hot field. They are very similar in a way to transformers because both do pairwise interactions between elements, the difference being that GNNs use explicit and transformers implicit graphs.
Re:Deep Graph Library: Easy Deep Learning on Graphs
by lmeyerov on Tuesday December 22, 2020 @10:24PM
(#25514051)
expanding on that: people used to do stuff like pagerank etc. ahead of time and decorate their tabular data with the results: entities, events, whatever for social, fraud, security, customer journey, whatever. that meant a phase separation between graph analytic enrichments and learning. bringing the non-local graph reasoning to the learning phase in a way that isn't slow enables closing the loop.
The early graphsage stuff was, afaict, proven for generic social recommendors, but most gnn's I see seem pretty custom (e.g., deepmind's protein folding solution), esp. when not prohibitively slow. It sounds like more generic use is becoming practical w/ these libs, and esp. interesting to me, the latest NIPS had graph transformers papers, which brings another level of practically here. Not sure if DGL & friends have those yet..
[1] https://github.com/yuehhua/GeometricFlux.jl
[2] https://fluxml.ai/
dgl is a library for graph neural networks (GNNs). The algorithms present in BG can be implemented in dgl, albeit much less efficiently but the reverse might not be true.
More specifically GNNs are a set of methods based on what is called "Message Passing" algorithm, where the embedding of each node is a function (parameterized over the model weights) of its neighborhood and the edges that connect the node to it.
Additionally GNNs target learning functions that work on multiple graphs for example graphs of molecules to predict their properties, not just a single graph.
dgl can be compared to PyTorch Geometric. The former works on both TF 2.0 and Pytorch while the latter is only for PyTorch.
Both are almost equivalent, although dgl has some institutions backing it. PyTorch Geometric might feel a bit more lightweight to integrate in existing codebases.
Does this thing learn a graph (i.e. build nodes and edges) from more unstructured data like text in order to make relationships explicit?
Does it take a knowledge graph and use it as partial input to help learn some other function?
Or something else?
Graph NN layer: f(X, A) = g(AXW), where g is an activation
Fully connected layer: f(X) = g(XW), like GNN where the adj. matrix A=I
To put it in context a few years ago GNNs became a hot field. They are very similar in a way to transformers because both do pairwise interactions between elements, the difference being that GNNs use explicit and transformers implicit graphs.
EDIT: found it! https://arxiv.org/pdf/1609.02907.pdf
[1] : https://nanonets.com/blog/information-extraction-graph-convo...
[2] : https://arxiv.org/pdf/1903.11279.pdf
The early graphsage stuff was, afaict, proven for generic social recommendors, but most gnn's I see seem pretty custom (e.g., deepmind's protein folding solution), esp. when not prohibitively slow. It sounds like more generic use is becoming practical w/ these libs, and esp. interesting to me, the latest NIPS had graph transformers papers, which brings another level of practically here. Not sure if DGL & friends have those yet..