News for hackers. Stuff that matters.
Re:Non-Posix File Systems by sillysaurusx on Tuesday September 08, 2020 @05:07PM (#24413383)
Surprised no mention of plan9's FS. It always seemed like the core innovation in plan9.

Basically, everything is a file. But to a ridiculous extent, far beyond what you'd normally think of as files. It's been years since I looked into it though, so maybe I'm misremembering.

Designing it that way has lots of advantages. For example, you can connect computers together via networks using the equivalent of `cat`. (And yes, we have netcat, but it's not quite the same thing as having the abstraction built into the OS.)

Re:Non-Posix File Systems by MayeulC on Tuesday September 08, 2020 @06:14PM (#24414188)
I am not entirely intimate with plan9, so take this with a grain of salt.

IIRC, everything was a file, down to the graphical UI. This allowed you to interact with the GUI trough the file system, script things, etc. Network connections and computers are also available trough files.

But even more impressive is that this filesystem builds on a relatively simple protocol[1]. Programs and computers can implement this to expose a filesystem view of their internals. The protocol is still used quite a bit, to share files across VMs for instance [2], and more generally when serializing filesystem data over a network link is needed.

I think this allows you to do fancy stuff such as remotely controlling a window on another device, or displaying it on an arbitrary number of machines.

I wish Linux used the filesystem that extensively, to replace things such as d-bus, pulseaudio, gobject, gstreamer, wayland, etc. GNU Hurd has a similar, very interesting mechanism, translators [3] (basically first-class FUSE objects).

[1] https://en.wikipedia.org/wiki/9P_(protocol)

[2] https://wiki.qemu.org/Documentation/9psetup

[3] https://www.gnu.org/software/hurd/hurd/translator.html

Re:Non-Posix File Systems by floren on Tuesday September 08, 2020 @06:21PM (#24414253)
Essentially yes. Here's how you get a graphical session on a remote machine: export your local /dev/draw, /dev/mouse, etc. via 9P to the remote machine, then run the GUI on that remote machine. It'll write to /dev/draw, read from /dev/mouse, and so on with no idea that all those file operations are actually traversing the network.
Re:Non-Posix File Systems by renox on Friday September 11, 2020 @04:25AM (#24440326)
Doesn't seem like a design usable for modern GPUs..
Re:Non-Posix File Systems by MayeulC on Tuesday September 22, 2020 @05:19PM (#24560061)
Thinking about it a bit more, indeed. Unless you compress the data of course, which could theoretically be done transparently by the graphical session that owns the framebuffer device.

But I think that /dev/draw (as the name implies) was likely a drawing API? A bit like X11, when the only thing GUI apps did was to draw primitives like circles, lines, text...

Re:Non-Posix File Systems by simias on Tuesday September 08, 2020 @06:16PM (#24414203)
If you look at the trend on "mainstream" OSs, it seems to be going the other way. Google and friends have been working hard to move the networking stack upwards into the userland, because they control the userland and not the kernel. I also blame Microsoft who sucks at providing timely and non-intrusive updates to their kernel, and since it's closed source nobody else can really do anything.

I always thought for instance that SSL/TLS ought to be implemented in the kernel, with an userland daemon dealing with certificate verification. That's how for instance Linux implements the spanning tree protocol: the actual bridging is fully done in-kernel and there's a userland daemon tasked with the higher level details of the protocol.

This way certificates stores and critical updates would be centralized in one location instead of having dozens of programs liking their own OpenSSL and using various certificate registries.

It's never gonna happen though. Oh well.

Re:Non-Posix File Systems by the8472 on Tuesday September 08, 2020 @07:06PM (#24414687)
> I always thought for instance that SSL/TLS ought to be implemented in the kernel

Linux provides TLS sockets. Only the bulk transmission is handled by the kernel. Handhsaking and renegotiation has to be handled by userspace. That allows offloading to hardware accelerators. In theory you can sendfile() from a NVMe drive through the a network card with crypto/compression handling without ever touching main memory with all the normal IO syscalls, i.e. with no userspace modification other than opening a TLS socket instead of a TCP one.

Re:Non-Posix File Systems by zie on Tuesday September 08, 2020 @10:26PM (#24416216)
FreeBSD does in-kernel TLS also. https://reviews.freebsd.org/rS351522
Re:Non-Posix File Systems by Mister_Snuggles on Tuesday September 08, 2020 @07:13PM (#24414754)
> I always thought for instance that SSL/TLS ought to be implemented in the kernel, with an userland daemon dealing with certificate verification.

This is (was?) available in Solaris[0]. I never used it, so I have no idea how well it worked.

[0] https://docs.oracle.com/cd/E26502_01/html/E28990/webk-3.html

Re:Non-Posix File Systems by non-entity on Tuesday September 08, 2020 @10:19PM (#24416151)
> If you look at the trend on "mainstream" OSs, it seems to be going the other way. Google and friends have been working hard to move the networking stack upwards into the userland, because they control the userland and not the kernel.

Any links on what they're doing with this, regardless of intention, I'm curious to see.

Re:Non-Posix File Systems by pjmlp on Wednesday September 09, 2020 @03:12AM (#24418188)
Most likely as a side effect from Project Treble, where Linux kernel is coerced into a pseudo-mikrokernel.

https://source.android.com/devices/architecture/modular-syst...

AIDL means Android IDL, which is the mechanism for Android IPC, also used by Treble for communication with modern driver model (classical Linux drivers are "legacy" drivers from Project Treble point of view).

Re:Non-Posix File Systems by floren on Tuesday September 08, 2020 @06:00PM (#24414035)
If you implement, for example, the network stack as a filesystem, and you have generic facilities which let you bind "files" from one system to another, you can forward your traffic through another host as easily as:

    import remote-host /net
Now your new network connections will go through remote-host, because a network connection is just reads and writes to files in /net.

Lots of neat applications when you take a few simple ideas and expand them as far as you can: represent everything as files in a namespace, provide advanced tools to manipulate that namespace, and do it all via a networked protocol.

Re:Non-Posix File Systems by Sebb767 on Tuesday September 08, 2020 @10:11PM (#24416081)
> Now your new network connections will go through remote-host, because a network connection is just reads and writes to files in /net.

Wouldn't this then also affect the connection to this remote host used for importing, effectively looping itself?

Re:Non-Posix File Systems by floren on Tuesday September 08, 2020 @10:38PM (#24416309)
Nope, because the connection to that remote host was established when we were using the local host's /net. This is one of the other key features of Plan 9: per-process namespaces (inherited by children). Only programs inheriting the current process's namespace will use the imported network stack: you open a shell, you run "import remote /net" to bring in the remote network stack, then you run your network applications within that shell. Applications in any other shell will use the regular local network stack. After importing /net from the remote host, you could also run `rio` in that shell, which will start a new instance of the GUI nested within that window, in which all applications will use the imported stack.
Re:Non-Posix File Systems by Sebb767 on Tuesday September 08, 2020 @10:49PM (#24416416)
Very interesting. Thanks for the explanation!
Re:Non-Posix File Systems by bear8642 on Tuesday September 08, 2020 @06:28PM (#24414322)
Indeed and that's what Plan9 enables you to do
Re:Non-Posix File Systems by floren on Tuesday September 08, 2020 @06:32PM (#24414368)
That's why I posted the exact command you would run on Plan 9 in order to do it ;)
Re:Non-Posix File Systems by ridiculous_fish on Tuesday September 08, 2020 @07:51PM (#24415001)
How did Plan 9 allow its APIs to evolve? For example, reading /dev/mouse returns records that are exactly 49 bytes, in text format [1]; how did they add new fields without breaking every app?

1: http://man.cat-v.org/plan_9/3/mouse

Re:Non-Posix File Systems by eadmund on Wednesday September 09, 2020 @08:15AM (#24420164)
As a sibling notes, they could have just added a new file while supporting the old format, but they had some leeway in the format itself: the buttons value only uses three bits.

Had they been a little more clever and used a delimited format like S-expressions, they could have simply specified that each mouse event would generate one S-expression, and clients could have read one expression at a time.

Re:Non-Posix File Systems by sedatk on Tuesday September 08, 2020 @08:07PM (#24415148)
They’d probably have /dev/mouse and /dev/mouse2 simultaneously.
Re:Non-Posix File Systems by saagarjha on Wednesday September 09, 2020 @06:14AM (#24419400)
I guess you could call it a stricter version of the Linux’s kernel’s ABI stability?
Re:Non-Posix File Systems by anthk on Tuesday September 08, 2020 @10:51PM (#24416429)
namespace
Re:Non-Posix File Systems by saagarjha on Wednesday September 09, 2020 @06:13AM (#24419392)
I’ve always wondered how Plan9’s “everything is a file” didn’t open up a billion race conditions. Did they solve this problem, or was this just something that was papered over?
Re:Non-Posix File Systems by amelius on Tuesday September 08, 2020 @05:32PM (#24413699)
Sounds cool, but at that point I guess you can stop calling it a file, and call it "object" instead.
Re:Non-Posix File Systems by anthk on Tuesday September 08, 2020 @10:52PM (#24416436)
If you can use read and write, it's a file.
Re:Non-Posix File Systems by Crontab on Tuesday September 08, 2020 @07:18PM (#24414799)
I think the thing that I found most interesting on Plan9 was using union directories to set the search path.
Re:Non-Posix File Systems by narwally on Wednesday September 09, 2020 @01:25AM (#24417455)
If this is what it sounds like would it be like making a /path directory and defining it as a union of all the directories containing executables. almost like an efficient and idiomatic way to create a single directory containing symlinks to all of your executables. How would it handle things like path precedence?
Re:Non-Posix File Systems by drybjed on Wednesday September 09, 2020 @07:39AM (#24419900)
That's exactly what it is, Plan 9 does not have $PATH. You just mount directories with binaries under /bin. When multiple binaries have the same name, the one from the last mounted directory is used, masking the others. You could have something like:

    mount /usr/bin /bin
    mount /usr/local/bin /bin
    mount ~/.local/bin /bin
Since binaries in Plan 9 are usually under additional subdirectories, conflicts are rare. For example instead of 'ping' you have 'ip/ping' which would be expanded to '/bin/ip/ping'. The same way binaries from different system architectures can live in the same filesystem, you just mount the directory for relevant architecture under '/bin'.

See also: https://en.wikipedia.org/wiki/Union_mount

Re:Non-Posix File Systems by eadmund on Wednesday September 09, 2020 @08:21AM (#24420197)
> If this is what it sounds like would it be like making a /path directory and defining it as a union of all the directories containing executables. almost like an efficient and idiomatic way to create a single directory containing symlinks to all of your executables.

That's basically it. Although 'defining' in this case really means 'binding directories into it.'

> How would it handle things like path precedence?

By ordering: first bind the main x86 binary directory into /bin, then bind your custom stuff ahead of it (or after it); as I recall, you could bind to the head or tail.

Re:Non-Posix File Systems by Crontab on Wednesday September 09, 2020 @12:30PM (#24422716)
That sounds right to me but it has been a while.