News for hackers. Stuff that matters.
Re:Cayley – An open-source graph database by jnordwick on Saturday April 08, 2017 @07:20PM (#14069608)
What exactly is this? The GitHub page speaks of different backends, and those appear to just be databases or key-value stores in themselves (e.g, Postgres and Bolt).

Is Cayley basically a query rewritter, that is it has some tables in the backend and when queried, Cayley then goes to the "real" (for lack of a better word) database? Cayley's query language might be more full featured, but it isn't a storage mechanism in itself?

There are two things from that:

1. There is no way for Cayley to take the graph structure of the data into account when laying it out on disk or when executing the query. Is this the long-term decision, or is this just a stop-gap until a storage mechanism can be done?

2. This would seem to imply that the abstraction layer from Cayley to the backend storage would be relatively slim. How difficult is it to add another storage driver for another SQL database or for one with a custom query language?

Another thing I noticed:

> query -- films starring X and Y -- takes ~150ms

Even on two year old hardware that seems dog slow - less than 7 queries a second - for a very simple query.

Re:Cayley – An open-source graph database by dimfeld on Sunday April 09, 2017 @02:44AM (#14070838)
Cayley's graph data layout is most similar to a Hexastore-style [1] triple store, though IIRC it doesn't do the full six-way index that the original Hexastore paper describes. The Redis page on secondary indexing [2] has a great quick intro to what this actually entails (search the page for Hexastore).

As you might guess from the Redis link, this style of graph lends itself well to KV stores, so the answer to your question #1 might be that it's a long-term decision, but the style of graph is really designed for a KV store anyway. But I haven't discussed this at all with the Cayley devs so I can't actually speak for them.

I'm using it with the BoltDB backend and have been pleased with the performance overall. I haven't looked at the backends for more complex databases like Postgres in detail, but it does appear that the backend interface has potential for predicate pushdown as well. The repository's graph directory [3] contains the various backends if you want to check it out. Overall it doesn't look very difficult to add another backend type, but I haven't tried it yet. Looking at the existing SQL backend, it appears to already support MySQL, PostgreSQL, and CockroachDB (but I've tried none of these with Cayley).

[1] http://www.vldb.org/pvldb/1/1453965.pdf [2] https://redis.io/topics/indexes [3] https://github.com/cayleygraph/cayley/tree/master/graph

Re:Cayley – An open-source graph database by itamarhaber on Sunday April 09, 2017 @07:54AM (#14071836)
Speaking of which, take a look at this Redis module that marries Hexastore and neo4j-like queries: https://github.com/RedisLabsModules/redis-module-graph
Re:Cayley – An open-source graph database by perfmode on Saturday April 08, 2017 @09:35PM (#14070010)
You can't derive rate from latency alone. The query might be IO bound. If that's the case, you can run queries concurrently.
Re:Cayley – An open-source graph database by jnordwick on Saturday April 08, 2017 @10:27PM (#14070149)
I tend to care more about latency than throughput. FedEx has the best throughput in the world if you can wait a couple days.

True though. I'm guessing with a latency number like that, the throughput is pretty bad too.

Re:Cayley – An open-source graph database by ashwin95r on Saturday April 08, 2017 @10:38PM (#14070168)
Exactly what https://dgraph.io is built for. Low latency, high throughput.
Re:Cayley – An open-source graph database by jnordwick on Saturday April 08, 2017 @10:57PM (#14070207)
Looks interesting.

That semi-declarative query language is hideous though. Reminds me of attempts to make XML into a programming language.

Re:Cayley – An open-source graph database by mrjn on Sunday April 09, 2017 @01:12AM (#14070606)
(author of Dgraph)

It's based on GraphQL, which is definitely not a hack like XML => language and currently catching on faster than a forest fire. Dgraph's derivation, GraphQL+- (for lack of better name) is a lot more powerful than Cypher or Gremlin. Both of the latter allow only returning lists of results; while GraphQL+- returns back an entire subgraph. Thus, all the relationships are maintained. It also allows expressing complex joins using a function like variable blocks; feeding results from one into another like you'd do in any popular language.

You can try running some complex queries with GraphQL+-, it might change your view. http://play.dgraph.io/

Re:Cayley – An open-source graph database by jnordwick on Sunday April 09, 2017 @02:29AM (#14070809)
Cool af

Edit: I still think it looks hideous. Def try to figure it out more tomorrow.