I am in the process of compiling this with emscripten to javascript+wasm. The integration between your block storage implementation and FatFs is straightforward other than ioctl, which is mysterious to me and will take me a long time to figure out what all my responibilities are.

This is my first time playing with emscripten and sharing buffers between wasm and javascript is awkward.

Sorry, but I can't help wonder: why?

What's the value of having direct device access in Javascript? Are we expected to give the browser raw block device access now? Maybe we could implement a FAT layer on top of S3, but I really don't see the value in reimplementing block-level storage on top of object storage.

Might be useful for non-browser VMs? E.g. a microcontroller running WASM.

Even tough yes, that's a good answer to the GP... Are people expecting WASM machines to be a thing? And with a different fate from the Java machines? (I don't see how they are different enough to change that fate.)

> Are people expecting WASM machines to be a thing?

People are expecting to embed WASM[1]. A generic FAT can provide a lightweight storage abstraction for such work. The WAMR AOT compiled runtime is 50K; 2.5-10% of the flash memory on a STM32F4, for example. Obviously not appropriate for the smallest extant MCUs. Otherwise it's a viable choice.

[1] https://bytecodealliance.github.io/wamr.dev/

so you can snoop on users pendrives if they make the mistake of allowing USB access from their google-chomeTM browser to your domain. ;)

I don't think Chrome allows access to storage devices through webusb.

I am working on a virtual computer for kids and retro loving adults. I wanted to simulate block storage using Local Storage as the backing store because I felt that blocks were very understandable. I've had block storage working for many years, but when I went to build a filesystem atop it, my design was uninspired and my implementation was buggy.

When I found FatFs, I realized I had a perfect fit: a realish filesystem with an API that I could map right onto system calls, and FAT32 - a klunky old jalopy that will need defragmenters and other disk tools to maintain it. It'll make for an altogether more interesting ecosystem and a gentle challenge for the user to maintain.

Also, there are many areas of my javascript that are primed for conversion to C -> WASM, like sprite collision detection. JavaScript optimizers have done ok with my code, but I'd like my compy to run fast on mobile too. If I am successful with FatFs, I will have a convenient spot in my codebase for converting my busy number-intensive routines into WASM.

There is a project doing some version of ext FS on top of S3. Just can't remember what it's called right now. It's active and on GH.

http://elm-chan.org/fsw/ff/doc/dioctl.html describes the expected ioctls to implement and their behavior. The mandatory ioctls seem pretty straightforward. CTRL_SYNC will be a nop on a browser-based storage, GET_BLOCK_SIZE should return 1 (unknown), and CTRL_TRIM looks like it could be nop (or you could zero the range to more closely match real devices).

I didn't study in my operating systems class! If only I could shake dumbass young version of me and have them work harder at school!

I know what you are saying is true and I should have no fear. I just need to write lots of tests and log like a mofo until I get all the ioctl commands it needs built out on the javascript side.

ioctl really is a ugly unixism that's totally out of place there.

It could have been replaced by 3 or 4 calls with a clear single purpose, and I'd hope they'll realize this and fix the API in a future version.

++ don't ioctl / sockopt unless you absolutely have to for backward compatibility. It's an old hazardous design. We have oodles of space for symbols even on modern embedded systems.

What is the recommended way under Linux then? It seemed like everyone was going full steam ahead with adding devices under /dev and /proc with control via read/write/ioctl. ioctl in my experience has been the best way to add a clean, modular interface to kernel modules since you don't need to hack in syscalls or do terrible, custom shared memory communication.

The context isn't Linux (an UNIX clone, ioctl is endemic there, no hope).

It is fatfs, a tiny fat library for embedded use, which most of the time will be used w/o an OS.

Netlink is becoming common, but it's also messy because well, it started as a single purpose interface and is now among other things a misnomer as it's not limited to network link controls.

kdbus could probably provide richer interfaces eventually, but proximity to freedesktop will probably perpetuate a certain amount of pushback.

The main point is that the ioctl interface is limited in its expression. It is hard to filter for security, and have bad evolutionary properties. Many of the subprotocol designs over the top of ioctl have memory safety hazards even beyond the immediate issue of needing to often make and trim buffers in less than ideal ways. This is to say the interface is hard to design well with, and a more formally typed system would provide a better solution.

Regardless, OP isn't on Linux, and the point is to not copy the bad interface!

It just works.

  > This is my first time playing with emscripten and sharing buffers between wasm and javascript is awkward.

You are using "SharedArrayBuffer", I assume?

I am preferring to use Uint8Array, Uint32Array or just let emscripten do the allocations for me. I haven't dabbled much with web workers, so I have avoided the whole transferrables thing.

There's an old an OS-free/BIOS-based implementation of FAT12 called MDCFS.C (Minimal Dos Compatible File System) in Dave Dunfield's EMBEDPC.ZIP (only in the Internet Archive now). You'll also need peek at DK86IO.C from that zip as well as come up with an implementation for those BIOS calls. Dave's been active on vogons lately.

https://web.archive.org/web/20210505023207/http://dunfield.c...

Thanks for the tip!

Can this do virtual files? For example if I turned a microcontroller into a false flash drive of sorts, and wanted to generate the contents of a file on the fly and not have them at rest in a FAT system.

This isn't really the tool for that. With FatFS, you provide a way to interact with the storage medium (disk read/write and so forth) and FatFS handles everything at the filesystem level (file open, directory listing, etc.) It would be much easier to emulate a filesystem (perhaps even copying parts of FatFS' API) than it would be to emulate a virtual disk and and access it at arm's length with FatFS.

From the website:

  FatFs controls the storage devices via a simple media access interface shown below.
  disk_status - Get device status
  disk_initialize - Initialize device
  disk_read - Read data
  disk_write - Write data
  disk_ioctl - Control device dependent functions
  get_fattime - Get current time

For your use case, you'd probably have to implement the disk_read method to dynamically generate and return your content based on the start and end sectors provided as parameters.