Re:Freqfs: In-memory filesystem cache for Rust
by haydnv on Friday October 01, 2021 @02:56PM
(#28722166)
One advantage is consistency across host platforms, but the main advantage is that the file data can be accessed (and mutated) in memory in a deserialized format. If you let the OS take care of it, you would still have the overhead of serializing & deserializing a file every time it's accessed.
[ Parent
| on HN ]
Re:Freqfs: In-memory filesystem cache for Rust
by dfranke on Friday October 01, 2021 @03:16PM
(#28722343)
That's what mmap is for.
Re:Freqfs: In-memory filesystem cache for Rust
by haydnv on Friday October 01, 2021 @04:01PM
(#28722773)
It might be possible to replace freqfs with mmap on a POSIX OS, but a) you would still have to implement your own read-write lock, and b) you would (I think probably?) lose some consistency in behavior across different host operating systems.
Re:Freqfs: In-memory filesystem cache for Rust
by vlovich123 on Friday October 01, 2021 @04:15PM
(#28722949)
Which OSes does this run on that doesn’t have some kind of mmap operation?
Re:Freqfs: In-memory filesystem cache for Rust
by haydnv on Friday October 01, 2021 @05:17PM
(#28723467)
It should work on Windows (because tokio::fs works on Windows) although I have not personally tested this
Re:Freqfs: In-memory filesystem cache for Rust
by julian37 on Friday October 01, 2021 @05:35PM
(#28723643)
You can do mmap on Windows, eg. https://github.com/danburkert/memmap-rs
Re:Freqfs: In-memory filesystem cache for Rust
by gpderetta on Friday October 01, 2021 @04:10PM
(#28722895)
mmaps for read, explicit API for writing, a-la LMDB. Buggy readers can read inconsistent data but cannot corrupt the os.
Re:Freqfs: In-memory filesystem cache for Rust
by otterley on Friday October 01, 2021 @04:39PM
(#28723181)
Corrupt the OS? How might that happen?
Re:Freqfs: In-memory filesystem cache for Rust
by gpderetta on Friday October 01, 2021 @06:30PM
(#28724066)
Sorry, I meant the DB!
Re:Freqfs: In-memory filesystem cache for Rust
by alexruf on Friday October 01, 2021 @03:23PM
(#28722419)
Personally I don’t see a scenario for myself, but I can imagine that there are some where this might be useful.
But isn’t there a extremely high risk of data loss an inconsistency when adding an extra layer on top of OS file system handling?
Re:Freqfs: In-memory filesystem cache for Rust
by ericbarrett on Friday October 01, 2021 @04:35PM
(#28723144)
Freqfs seems like a shim you'd add to an existing project for a quick optimization. Whereas mmap et al. are "better" the same way any specific, built-to-purpose code will be "better" than just bolting a framework on. Sometimes it's the right call to do the extra work; sometimes it's 100% more effort (both development and maintenance) for an extra 10% gain.