Re:How Streak built a graph database on Cloud Spanner to wra...
by sandGorgon on Sunday November 04, 2018 @03:50PM
(#18377998)
Would love to know more details about the graph traversal/fanout part. This is the stuff that most of us go to spark for, so would love to know how spanner makes this easier.
Will put together a follow-up with some more information, but in summary: * yeah, no built-ins for graph operations in Spanner other than relational SQL for standard joins * the key thing that make this work are the ability to easily construct global indexes that aren't sharded by the primary key and reasonably fast joins between them * it's also helpful that Spanner does a reasonable job of parallelizing queries (e.g. a lot of times we'll get a 15x increase in speed vs. a sequential plan) * we then do the fan-out across the graph in our Java Spanner client - each distributed SQL index read takes ~10 ms so we can do multiple round trips of graph traversal in the client