> Beyond the ability to unambiguously specify which process should be waited for, this change will eventually enable another interesting feature: it will make it possible to wait for a process that is not a child — something that waitid() cannot do now. Since a pidfd is a file descriptor, it can be passed to another process via an SCM_RIGHTS datagram in the usual manner. The recipient of a pidfd will, once this functionality is completed, be able to use it in most of the ways that the parent can to operate on (or wait for) the associated process.

So, to wait for a process that is not your child, do you have to get the relevant pidfd from its parent? In which case, this doesn't help all that much. Or is there some other way to get pidfds for arbitrary processes?

Perhaps pidfd_open(pid, ...)? https://lwn.net/Articles/789023/

I find it bizarre they called it "pidfd_" rather than just "process_" or "proc_"... almost seems like they deliberately avoided the obvious?

It's because the object you get is a file descriptor.

In fact it's exactly equivalent to getting a file descriptor for /proc/$pid -- Christian (the person who developed the patchsets) quite cleverly solidified a trick that some folks knew about for several years (that you could use /proc/$pid as a race-free way of checking if a process has died if you grabbed a handle while it was still alive). Before pidfd_send_signal(2) there wasn't a way to use that "interface" nicely. But now it's a first-class citizen (and Christian had to fight a lot of battles to get this in over several releases).

It's really cool work and I have high hopes it will be used far and wide because it solves so many individual problems in one fell swoop.

> It's because the object you get is a file descriptor.

It is, but you're opening the object, not the descriptor (which doesn't even exist yet). When you open a kernel object, you always get back a (new) file descriptor representing that object. It's against previous naming conventions too. It's not like mkfifo() was called mkfifofd() or socket() was called socketfd() or perf_event_open() was called perf_event_fd_open()...

I'm really amused that you're so excited about it and find it so cool. I mean, I'm not suggesting it isn't awesome that they added it, but to me, it's such a glaring obvious deficiency that I'm completely flabbergasting that a lot of battles had to be fought to include it. It should've been added and embraced with open arms over two decades ago...

> It is, but you're opening the object, not the descriptor (which doesn't even exist yet).

pidfd_open(2) is still a proposed interface, and isn't in mainline yet (and if I'm remembering the ML discussions correctly, it might not even go in any time soon). The currently-available interfaces are pidfd_send_signal(2), CLONE_PIDFD, and the new pidfd_poll(2) support for pidfds. In that context, calling it "pidfd_" makes more sense (since you're operating on existing handles) and thus pidfd_open(2) also makes sense because otherwise the naming would be needlessly inconsistent.

There are also several pre-existing APIs that are called process_ (such as process_vm_{read,write}v(2)) which use pids and not pidfds -- so calling the new APIs process_ (or even proc_) could lead to confusion. From memory the first couple of iterations of the patchset changed the name several times until we landed on pidfd_ and nobody really complained much afterwards.

Also (and now I'm just nitpicking), mkfifo(3) doesn't give you an fd -- it's just a wrapper around mknod(2). But I do get your point.

> I'm really amused that you're so excited about it and find it so cool.

I might be a little bit more biased towards thinking it's cool, since the developer is a good friend of mine and we went back and forth on the design quite a lot (the fact he managed to get /proc/$pid fds to have these features is pretty remarkable and it's unbelievably cool that it didn't require having multiple classes of fds -- if you'd have asked me a year ago I would've said it'd be very hard to get right and would never be merged because it'd be so invasive). But thinking that it's neat isn't mutually exclusive with thinking that (something like) it should've been implemented a long time ago.

What obvious? The command creates and returns a pidfd for the pid passed, not a process or a proc (which would be conflated with other concepts in Linux).

But you're opening the object (which already exists), not a file descriptor (which doesn't exist yet). And when you create or open a kernel object, you always get back a (new) file descriptor representing that object. See perf_event_open, mkfifo, etc. I'm not really sure how it'd have conflated with anything to call it proc_open or something like that, but I'll take your word on it.

You can get a pidfd by opening /proc/$pid. That's all a pidfd is. The proposed pidfd_open(2) is for separate and more specialised use-cases where /proc isn't available and other restrictions apply.

Posted 1 week ago! Do you know what took them so long to realize this is the correct way to do it? Windows has had it for some two decades...

Unfortunately, as Linux has matured, many of its developers refuse to adapt technologies for the sole reason that it "stinks" of Microsoft.

Cutler and his team were pretty forward thinking given that NT is over 30 years old. See also: the async i/o headaches and the inability to WaitForSingleObject()/WaitForMultipleObjects() (or an analogue) in Linux.

It's a shame really.

> many of its developers refuse to adapt technologies for the sole reason that it "stinks" of Microsoft.

I don't where did you get that idea but it is not true at all. Linux developers have a history of ignoring (and reinventing) technologies and interfaces developed in other Unix systems, too. You can listen any Bryan Cantrill rant for more details. The claim that they refuse to adapt technologies that "stink" Microsoft is a mischaracterization.

re: WaitForMultipleObjects, finally something like that is likely coming to Linux:

https://www.reddit.com/r/linux/comments/ck77gm/valve_propose...

> It's a shame really.

Well, yes, and of course, yet at the same time it's FOSS, so ... if someone really needed it, they should have proposed a patch. ¯\_(ツ)_/¯

> Well, yes, and of course, yet at the same time it's FOSS, so ... if someone really needed it, they should have proposed a patch. ¯\_(ツ)_/¯

It's a bit like doing I/O that doesn't throw your data away... just use O_DIRECT, bring your own I/O manager and page cache and scheduling and simply implement all the low-level compatibility stuff yourself if you need such a weird thing ¯\_(ツ)_/¯

I mean MySQL did that with libaio (which needs O_DIRECT anyway), Oracle on Linux did that and some with ASM (their own filesystem basically), also Ceph with BlueStore (LevelDB as a FS).

Because general purpose filesystems and I/O susbsystems are just that, general purpose.

PostgreSQL regularly woes about the problems of the Linux I/O and fs APIs, but ... to my knowledge they did not do much else. (Other than working around the problems in userspace, and taking the performance hit.)

If you want I/O that doesn't throw you data away, use the perfectly fine buffered I/O on let's say XFS. Fast, safe, sane. If you need more juice, add NVMe, and/or MD-RAID / LVM. And eventually you need to scale out to multiple nodes anyway, and then the performance of the cluster consistency subsystem will be the weak point.

Things don't get accepted into mainline just because you did the work. This is a weird misconception people have about open source. You also see it when people ask "why did you build this instead of just contributing it to $largeProject?"

Huh. I never said it'll be magically accepted.

But you also can't say that it's somehow impossible to get anything other than what Linus et al. think is the very epitome of perfect.

Upstreaming stuff is an effort. Contributing to such a complex and fragile pile of code/crap/C must be done through the kernel devs.

But if you want to add a new FS, I/O scheduler, bytecode VM, or whatever, it's possible. Look at Ceph, btrfs, bcachefs, all the SSD/NAND oriented FS-es, look at eBPF, and the XDP [eXpress Data Path] IO Visor network thingie, or WireGuard [which is still not merged as far as I know, because it tried to bring its own crypto library instead of using the already existing stuff]. All the stuff getting upstreamed from Android piece by piece.

A lot of kernel devs are happy about this or that, but they are pragmatists.

And Linus doesn't veto big ideas just because, but does not want to pull sub-par implementations. (See KDBUS for example.)

RHEL had an entire async i/o subsystem. It was rejected upstream.

Could you provide some details about this?