With a generic, system wide api there’s no reason you couldn’t still poll if you really wanted to. But maybe at least with a version token. So the application can say “I want info on which processes are running. Last time I asked, you gave me token 5827275” and the kernel can respond and say “you’ve got the most recent version of that data already champ”.
Think about it - the kernel stores some data in a nice, neat struct. And the kernel knows exactly when that struct is modified. Userspace wants that data, and wants to know when the data changes. So every second, userspace "reads" a pseudo file. The kernel packs the struct's contents into a big string with some sloppy, hand written text format and passes it to userspace. Then in userspace you need to write a brittle parser for whatever the kernel gives you, knowing that in different versions of the kernel you'll get slightly differently formatted strings.
When the data changes in the kernel, the application has no way of being notified. It just polls and parses in a loop. Its both inefficient (when there are no changes, its all wasted effort). And its laggy (when changes happen, it'll take your application up to the polling interval to find out about the changes).
For things like filesystem watching, there are special APIs applications can consume. But those APIs are different for every kind of data the kernel manages. Devices, mount points, USB connectivity, filesystem watching, CPU usage, thermal sensors, open network connections and on and on. All of this stuff has totally different kernel-level APIs for "watching" changes. (If you're lucky - a lot of it can only be polled). Its a big fat mess. I want to try replacing all of that with a nice clean API for fetching & subscribing to any kind of data. So, the kernel has a standard API for fetching (maybe it returns JSON). And a standard API for subscribing to changes - such that your application is notified when a change occurs, and the change is sent in a standard, easily digestible patch format.
And ideally, eventually, I'd like applications to be able to share their own changes back with the kernel in the same form & format. And maybe, share data objects between applications like that - with the kernel acting as a broker for sending patches back and forth, and syncing them with the filesystem. I think that would be really neat and useful for all sorts of applications.
But I want to prototype it all first in something thats not linux. I'm just not sure where to start. I don't really want to write an OS from scratch just to try out this generic state passing idea.
PS /proc isn’t a file system, it is an api (that can be accessed by the file system)
Windows has WMI and other systems have SNMP MIB as a uniform interface to access system information. Those never get anyway.
My expectation is performance would be better because you don’t need to parse the data as a string.
And you also avoid the ABA problem this way. If you wait for a file in procfs to change, by the time you read it it may have changed to something else - and you’ve lost data.
I wrote a unix-like kernel from scratch long back. It was not a fully functional unix but had a lot of components ready to the extent it would provide a command line interfaceI from which you can run commands like ls, cat etc by loading elf executables.
I would highly recommend to read this tutorial http://www.osdever.net/bkerndev/Docs/title.htm
Feel free to reach out to me (email in my profile page) if you want to discuss the topic with me.
Thanks for the kind words! I'll echo rajeevk's recommendation: the OSDev wiki is full of useful documentation and guides for setting up a bare-metal toolchain. There's also a community forum with a wealth of knowledge just waiting to be read.
In terms of boots on the ground, I remember once seeing a mantra from someone on the OSDev forums that went something like this: building an x86 OS is first about getting the CPU into a usable state, then about getting as far away from it as possible. This is really true! There's a lot of work to do to set up the features that any modern environment needs as a baseline: paging, segmentation, interrupt descriptors, etc, but once you're done with all that you're free to roam in the design space and build whatever you like, however you imagine, building abstractions and then abstractions on top of those.
I'd liken the early stages of OS development to a tech tree in a video game: you need serial output so that you can debug your kernel. You need to be able to allocate memory so that you can create pixel buffers, then need drawing primitives to fill them up. You need processes, and a scheduler to coordinate them. Every new feature you add opens up a myriad of possibilities that weren't there before, and although many things have a serial list of dependencies, there's always so much to do that you can just focus on whatever feels particularly inspiring to you at any given time. It's really a lot of fun.
axle also uses structured messages for making requests to the kernel, as well as everything else. Here's an example of some structured requests that can be sent to the kernel: https://github.com/codyd51/axle/blob/paging-demo/kernel/kern.... Here's some structured data that an application (in this case, the window manager) uses to interact with other apps: https://github.com/codyd51/axle/blob/paging-demo/rust_progra...
All the best, and feel free to reach out to me as well!
I really just have one idea I want to explore, which is this idea around sharing state between the kernel and userland via subscriptions and patches. (By taking some ideas from event sourcing and things like that). I guess I'm trying to figure out where I would even start prototyping something like that. I guess I could just fork axle and hack on it a bit to see how it goes?
I’ve had an idea I’ve been wanting to build into an OS for ages - where we use versions, message passing and structured patches to pass information between the kernel and application processes. So an application can subscribe to a file / USB device state / list of running processes / anything using a common, standard API. But it sounds really daunting to try and hack something like into the Linux kernel or something in one go.
If I want to experiment with something like this, what do you recommend? How hard would it be to do something like that in a hobby kernel like Axle?