News for hackers. Stuff that matters.
Re:Freqfs: In-memory filesystem cache for Rust by arghwhat on Friday October 01, 2021 @03:48PM (#28722636)
Your filesystem is a database. It's just a document-oriented database, rather than relational SQL.
Re:Freqfs: In-memory filesystem cache for Rust by cogman10 on Friday October 01, 2021 @04:06PM (#28722848)
It even has a lot of the same features as a full fledged DB.

For example, most file systems today are journaling. Which is exactly how most databases handle atomic, consistent, and durability in ACID.

About the only thing it's missing is automatic document locking (though most file systems support explicit locks).

That said, there are often some pretty hard limits on the number of objects in a table (directory). Depending on the file system you can be looking at anywhere from 10k to 1 billion files per directory.

There are also some unfortunate storage characteristics. Most file systems have a minimum file size of around 4kb, mostly to optimize for disk access. DBs often pack things together much more tightly.

But hey, if you can spin using the FS as a DB... Do it. Particular for a read heavy application, the FS is nearly perfect for such operations.

Re:Freqfs: In-memory filesystem cache for Rust by gpderetta on Friday October 01, 2021 @04:08PM (#28722866)
The biggest problem is the lack of good transactional facilities.
Re:Freqfs: In-memory filesystem cache for Rust by the8472 on Friday October 01, 2021 @05:20PM (#28723499)
you can lock directories, you can atomically swap directories (on linux), CoW filesystems make cloning kind of cheap. That could be used to implement transactions and commits. Getting the consistency checks/conflict detection during the commit right would be the most difficult part. Change notifications could be used to do some of that proactively. It's a terrible idea, but it could be done.
Re:Freqfs: In-memory filesystem cache for Rust by wongarsu on Friday October 01, 2021 @06:50PM (#28724244)
There is a transactional API for NTFS in Windows [1]. It allows transactional operations not just within a file but also across files or across multiple computers (to make sure something is applied to your whole fleet atomically).

1: https://en.wikipedia.org/wiki/Transactional_NTFS

Re:Freqfs: In-memory filesystem cache for Rust by cogman10 on Friday October 01, 2021 @04:17PM (#28722968)
Yup, the I in ACID is a bitch :)
Re:Freqfs: In-memory filesystem cache for Rust by edoceo on Friday October 01, 2021 @04:50PM (#28723271)
You mean it's always been NoSQL?

Astronaut with gun: always has been.

Re:Freqfs: In-memory filesystem cache for Rust by amelius on Friday October 01, 2021 @06:10PM (#28723932)
It doesn't really support transactions very well, though.
Re:Freqfs: In-memory filesystem cache for Rust by arghwhat on Saturday October 02, 2021 @03:25PM (#28731250)
True, but that's not at all a requirement for a database, and MVCC can be built on top where needed.