News for hackers. Stuff that matters.
Re:PlentyFS: PoC read-only filesystem with random contents g... by dredmorbius on Monday October 19, 2020 @02:20AM (#24823758)
The inspiration for this was wanting to develop and test file access and backup tools on many large files, without actually requiring the physical storage for origin data, and minimal memory or system overheaad.

Files are regular files (not FIFO's, named pipes, sockets, or symlinks), and data are random (not simply ASCII null as with /dev/zero).

Access is read only.

To be clear, this is a test rig, not for actual data storage.

Re:PlentyFS: PoC read-only filesystem with random contents g... by rwmj on Monday October 19, 2020 @03:27AM (#24824023)
Some comments: The seed should be an optional parameter (I see there's a comment in the code mentioning this). This is very important for CI testing where a bug only happens for specific file content and you need to reproduce the bug locally.

Why use a cryptographic hash? For nbdkit-random-plugin we use a fast PRNG (xoshiro256-starstar 1.0 as it happens, but the specific one is not important). I doubt a cryptographically secure hash is worth the extra computational cost to users.

Re:PlentyFS: PoC read-only filesystem with random contents g... by minoru on Monday October 19, 2020 @03:46PM (#24830145)
Hi, the author here. Thanks for taking a look at the code!

I picked SHA-1 because it was the simplest way to generate pseudorandom data: I knew there's a Rust crate for it which I used before. Works fine for a proof-of-concept :) I'm now reading up on other hashes and ciphers. My tentative plan was to replace SHA-1 with AES-NI in counter mode, but if I can find something that works on all CPUs while giving me a comparable performance, I'll go for that. (Should probably look at PRNGs as well; if I concatenate the file seed with the counter and use that as a seed for a PRNG, then ask it for a value -- it should be good enough too.)

I doubt Lars has requirements for the quality of randomness, but we'll see.

Re:PlentyFS: PoC read-only filesystem with random contents g... by dredmorbius on Monday October 19, 2020 @02:13PM (#24829119)
I'm really just a bystanderhere. File an issue w/ the project.