News for hackers. Stuff that matters.
Re:Diskhash – Disk-based, persistent hash tables by rkv on Saturday July 08, 2017 @06:50PM (#14727576)
Nice and simple. I tend to use a wrapper around BSD tree.h and SQLite for persistence. Longer insert/update times but quick access and you get the SQL language and tools for free.
Re:Diskhash – Disk-based, persistent hash tables by fnl on Saturday July 08, 2017 @07:26PM (#14727689)
Quick access? Tree lookup is O(log n) on a rb-tree. Hashtables are O(1). And this Diskhash thing claims to scale to billions of entries. Not sure your options are comparable, therefore.
Re:Diskhash – Disk-based, persistent hash tables by fleitz on Saturday July 08, 2017 @10:47PM (#14728120)
Claims is the operative word. Features generally aren't free. If mmap'ing files was some sort of performance godsend don't you think most DB manufacturers would just rip out all their custom caching code and replace it with mmap?
Re:Diskhash – Disk-based, persistent hash tables by fnl on Sunday July 09, 2017 @03:56AM (#14728828)
Absolutely correct! I guess a comparison with various levels of "rows" and concurrent access might be needed before judging any way...
Re:Diskhash – Disk-based, persistent hash tables by adrianN on Sunday July 09, 2017 @05:30AM (#14729001)
log(billions) is not very large, constant factors matter in practice. You can also do things with trees that are very difficult with hashes (e.g. ordered traversal). This is why databases still ship with tree indices.