mapleeman

So is it because of C++ vs Java? Well, not everything can be written just to these differences, a lot of stuff can influence results, architecture, database isolation level etc. One of the reasons is definitely a C++ and Java argument. Take a look at memory usage, here, you can see how JVM is memory hungry. Also, it takes time to warm JVM and Neo4j, so it is a mother penalty for the same reason.

So far, on this dataset and scale, we didn't encounter but we have plans for bigger dataset and more complex queries, you can take a look at the limitations part of this benchmark.

What are the downsides, Memgraph and Neo4j are currently a bit different vendors, Memgraph is an in-memory database, while Neo4j is on disk. So in Memgraph's case you are exclusively using RAM as a storage but gain speed, we have snapshots for disk for recovery, etc. While Neo4j is on disk, not-in memory but they are loding a bunch of stuff in RAM and consuming waste amounts of RAM, so it is hard to give pure distinction.

Yes, testing and benchmarking is a full-time job! The extra issue here is making it work under a single client to minimise latency penalties across measurements, then, there is also a protocol issue. Nice project with hash-db, I guess it is quite the learning experience(distributed-multimodal)?

Haha, adding to a backlog of things to do, add support for TigerGraph https://github.com/memgraph/memgraph/issues/689

Hey, thanks for the comment, you are 100% right, this is just the initial version since we are compatible with Neo4j, so it was a least effort to do it. It is just the initial setup, making it language agnostic will take a bit time. If you peak at the methodology and future part: https://github.com/memgraph/memgraph/tree/master/tests/mgben... You will see that we have the plan to add more database vendors + make it language-agnostic. We are also keeping track of all comments regarding this, I have opened an issue: https://github.com/memgraph/memgraph/issues/689, there is a language agnostic note in there. If you have any other input, it would be highly appreciated.

Hi everyone! I’m one of the co-authors of BenchGraph[1], a platform for Graph Database Performance Benchmarks. Our platform shows the results of running benchmark tests (via mgBench) on supported vendors. It shows the overall performance of each system relative to others.

Inspiration came from ClickBench, a Benchmark For Analytical DBMS.

We previously developed mgBench as in-house testing infrastructure to benchmark Memgraph, and now we are adapting it to support other graph database vendors. In order to test graph database performance, mgBench executes Cypher queries on a given dataset. Queries are general and represent a typical workload that would be used to analyse any graph dataset. Running this benchmark is automated, and the code used to run benchmarks is publicly available. You can run mgBench yourself to validate the results on the BenchGraph platform. The methodology is explained in detail on GitHub repo [2]

As you can see, at the moment, we have two vendors on the platform. We would like to add more vendors to our platform. If you want, feel free to contribute.

Let me know if you have any questions or suggestions.

[1] https://memgraph.com/benchgraph [2] https://github.com/memgraph/memgraph/tree/master/tests/mgben...