News for hackers. Stuff that matters.
Re:Scaling nodes in a Dgraph cluster by trishume on Tuesday June 21, 2016 @09:08AM (#11945310)
Thanks for the info. I guess my follow up question is why doesn't Dgraph end up caching all the data on every node when it has the memory to do so? Is this a planned feature that you just haven't implemented yet?
Re:Scaling nodes in a Dgraph cluster by mrjn on Tuesday June 21, 2016 @09:23AM (#11945387)
It depends upon the memory available. We have a background goroutine which checks how much memory we're using and evicts posting lists if we exceed a predefined usage. If we have a lot of memory then we just won't evict the posting lists, which means they'll be in RAM.

All this "caching" happens close to the disk level, just above RocksDB. We don't cache any results -- to me caching is cheating -- and we want to build something truly low latency. A user can easily add caching if they want to decrease the load on Dgraph, but it's not something I think we should be doing at the db level.