edit: the model layer is the most impressive part about this. You should consider making it a stand-alone package.
* loading too many edges (10K+) and associated nodes will be slow. * Traversing nodes in a meaningful way can be difficult.
To solve these, the schema has the following indices:
On edge table: (`from_node_id`,`type`,`updated`) On node_data table: (`type`,`data`(128))
Since edges rarely change, the first index allows you to paginate over edges by using updated as the order. As long as you request a reasonable number of edges, things should work OK. The second index is needed to get a node given some data, but a secondary use is sorting. By precomputing some score and saving it in node_data you can traverse nodes in that order (this is not currently built but is simple to do in SQL).
All this being said, the schema is pretty index heavy so if MySQL is forced to kick some of those out of memory it may lead to a bad time.
Thanks for the kind words on the model, I never thought about it being separate, but it makes 100% sense.
When I say no magic, what I mean is how you can understand what data is being loaded every step of the way so the mistake I mentioned previously is harder to make.
Also, I wanted to avoid query joins for easy sharding (if needed), but that adds an extra round trip so if needed the developer can write their own joins (giving up a bit of flexibility in the process).