News for hackers. Stuff that matters.
Re:In-memory vs. disk-based DB: Why do you need a larger tha... by CraigJPerry on Saturday September 02, 2023 @02:58PM (#37364417)
I think i’m missing something here, could you not just mmap() the file in and let the kernel take care of memory pressure for you?
Re:In-memory vs. disk-based DB: Why do you need a larger tha... by hakunin on Saturday September 02, 2023 @03:03PM (#37364472)
I'm no systems programmer, but I remember trying to research mmap approach. It's been 3 years, so I'm not sure what stopped me, but something didn't feel right. Perhaps it was the lack of control over how memory is used. I could clearly see how not use it, and didn't want any fluctuations.

Edit: oh and I think I did come across some article like the one linked in the neighbor comment. It's starting to come back.

Re:In-memory vs. disk-based DB: Why do you need a larger tha... by sschnei8 on Saturday September 02, 2023 @03:02PM (#37364460)
Re:In-memory vs. disk-based DB: Why do you need a larger tha... by rubiquity on Saturday September 02, 2023 @03:17PM (#37364591)
Good paper and overall the right opinion but not very relevant to a relatively simple system as OP described since they weren't dealing with safety, transactions, etc.
Re:In-memory vs. disk-based DB: Why do you need a larger tha... by legulere on Monday September 04, 2023 @10:40AM (#37380904)
Also not access times, so OP doesn’t care if the OS frees the pages again.
Re:In-memory vs. disk-based DB: Why do you need a larger tha... by lisper on Saturday September 02, 2023 @06:02PM (#37365902)
Sure -- until your system crashes and data is left in an inconsistent state because some of it was written to non-volatile storage and some of it wasn't and your OS had no concept of transactional consistency because mmap is a leaky abstraction.
Re:In-memory vs. disk-based DB: Why do you need a larger tha... by posnet on Saturday September 02, 2023 @06:32PM (#37366084)
Not a problem here. Read the top level comment.

"static key-value data. ... a const (i.e. a hash table)"

Re:In-memory vs. disk-based DB: Why do you need a larger tha... by lisper on Saturday September 02, 2023 @07:01PM (#37366218)
Ah. I stand corrected.
Re:In-memory vs. disk-based DB: Why do you need a larger tha... by legulere on Monday September 04, 2023 @10:41AM (#37380920)
And Flatbuffers for data-layout and access.