News for hackers. Stuff that matters.
Re:Show HN: We have built a benchmark platform for graph dat... by mapleeman on Thursday December 01, 2022 @09:16AM (#33816470)
Yes, good point, but both Neo4j and Memgraph(https://memgraph.com/product) are ACID-compliant and have on-disk persistent storage. Memgraph is currently RAM constricted, while Neo4j is not but it is hungry for RAM.
Re:Show HN: We have built a benchmark platform for graph dat... by buremba on Thursday December 01, 2022 @09:30AM (#33816662)
Is the comparison using on-disk persistent storage of Memgraph? The facts that they're both ACID-compliant and you can backup the data to disk in Memgraph are not relevant IMO considering the fact that I/O is the main driver for the latency for most of the workloads and reading data from disk is much slower compared to memory. Not sure how Neo4j works but an alternative might be running Neo4j in-memory mode.
Re:Show HN: We have built a benchmark platform for graph dat... by mapleeman on Thursday December 01, 2022 @04:06PM (#33822755)
No, currently, Memgraph is loaded 100% in RAM. Agree with the point about latency regarding I/O. The story with Neo4j is a bit more complex one. They are loading graph in memory, and using disk storage, doing both. But take a look at memory usage from them, it is the magnitude higher then Memgraph.
Re:Show HN: We have built a benchmark platform for graph dat... by menaerus on Friday December 02, 2022 @05:43AM (#33828870)
Did you manage to figure out where does the difference in memory consumption come from? Do you attribute it only to JVM or have you designed low-memory footprint data structures in memgraph?
Re:Show HN: We have built a benchmark platform for graph dat... by buremba on Friday December 02, 2022 @07:17AM (#33829552)
Java programs tend to use the memory they're given to provide the best performance in the server environments so it's not conservative in terms of memory allocation. While it does use more memory in a given time, it doesn't necessarily mean that it relies on all the allocated memory. That's one of the reasons why benchmarking is so hard..
Re:Show HN: We have built a benchmark platform for graph dat... by mapleeman on Friday December 02, 2022 @07:36AM (#33829714)
Exactly, you are right.
Re:Show HN: We have built a benchmark platform for graph dat... by mapleeman on Friday December 02, 2022 @07:41AM (#33829745)
The difference comes from several things, JVM being the first and obvious one. It takes as much memory as whole Memgraph + small dataset in this case. Second is the overallocation that JVM/Neo4j is doing, taking a bit more memory to have future space to grow. The third one comes to the implementations of storage and Neo4j cache implementation, not sure what Neo4j does on that side and how impactful it is. We use a skip list.