News for hackers. Stuff that matters.
Re:How do groups work on Linux? by laumars on Tuesday November 21, 2017 @12:07PM (#15749400)
> If you were to demonstrate that it's universal

I really wasn't. I was just making a point that it's not just Linux-specific like you claimed. In fact if you re-read the thread you'd realise that nobody argued this behaviour was universal. Given the breadth of POSIX-like systems out there, I doubt anyone would dare make such a bold statement. So you're arguing against point that was never raised.

> And by the way, filesystems are typically implemented in kernel, so saying that it's not kernel that handles them is somewhat misguided.

Well yes file systems generally are, except when they are not. For example ntfs-3g (which is also compiled against FUSE). Plus the file system code is modular - not in the monolithic vs microkernel sense but rather in terms of the body of code is a self-contained chunk that can be optionally compiled in or not. This also means that some file systems are written with some degree of portability in mind (easier said than done I know!!!)

What the argument is about is whether the permission logic described (specifically the setgid bit on folders) is handled by the file system code or externally to the file system code in other parts of the kernel (thus the behavior is defined by the OS rather than the file system).

You're making the case that the behavior is independent to the file system implementation (thus the logic is handled outside the file system code) because it is OS-specific. I'm stating that when I've been writing file systems I've had to handle that logic inside the file system code itself thus the logic is not OS-specific but rather file system-specific.

Now there will be standard way of implementing the logic because people will have certain expectations about how the file system should behave. Much like most $SHELL's broadly follow a common POSIX standard. So you would get the illusion of the discussed behavior being OS-specific even though the actual business logic is implemented at a different stage in the stack (ie the file system code). There might also be instances where one OS might patch the file system code to change its behavior to match the expectations of that OS - but I'm just speculating here. All I know is that I've had to write the logic myself - the kernel did not handle that for me.

I'm obviously not boasting to be a low level kernel hacker so I'd happily bow to a more reasoned argument but the examples you've given have just been guesswork about kernel operations based on some flawed experimentation with user space tools (I say 'flawed' because you didn't even bother to set the setgid bit in your original example when you suggested FreeBSD didn't support said behavior).

> Hint: what's the ownership of a file created on FAT mounted under Linux? It should suggest you what part of the kernel needs to handle the permissions.

That doesn't prove either of our points. Nor anything aside that Linux can mount FAT file systems.

Re:How do groups work on Linux? by dozzie on Tuesday November 21, 2017 @12:32PM (#15749649)
>> If you were to demonstrate that it's universal

> I really wasn't. I was just making a point that it's not just Linux-specific like you claimed.

I never claimed it's Linux only, only that there are other systems with different semantics.

>> And by the way, filesystems are typically implemented in kernel, so saying that it's not kernel that handles them is somewhat misguided.

> Well yes file systems generally are, except when they are not. For example ntfs-3g [...]

...except that FUSE is in-kernel mechanism to delegate VFS callbacks to userspace program. It's hard to argue in such a case that the work is not the kernel's responsibility.

> What the argument is about is whether the permission logic described (specifically the setgid bit on folders) is handled by the file system code or externally to the file system code in other parts of the kernel

Oh. This explains quite a bit, because I thought the argument was about whether the OS' usual semantics (group being inherited from directory or from EGID) are like that or not, not where in kernel this behaviour is coded.

> You're making the case that the behavior is independent to the file system implementation [...]

No, I'm making the case that Linux usually works one way, and BSD usually works the other way. I never claimed that anything was coded in any specific kernel subsystem, so don't add the statement yourself.

Re:How do groups work on Linux? by laumars on Wednesday November 22, 2017 @04:28AM (#15755508)
We are going round in circles so I'll have one last attempt to explain my point :)

> I never claimed it's Linux only, only that there are other systems with different semantics.

Maybe you shouldn't have said "Linux-specific" if you didn't mean it was specific to Linux.

Also I demonstrated that FreeBSD + ZFS behaves the same as Linux on my system. I cannot comment about why your example didn't work the same. Perhaps you were using UFS instead of ZFS? Or non-default file system mount options / or other tweaks? Aside running ZFS as root and having compression enabled, my root zpool is pretty vanilla. But either way, it goes back to my point that this is a file system specific behavior not an OS-specific one.

> ...except that FUSE is in-kernel mechanism to delegate VFS callbacks to userspace program. It's hard to argue in such a case that the work is not the kernel's responsibility.

Your counterargument is just reiterating my point. File systems compiled against FUSE do not run in kernel space. The kernel syscalls obviously need to hook to your file system still (which is where the FUSE wrapper comes into play), but the FUSE API does all the user space / kernel space memory swapping so your actual FUSE file system doesn't run in the kernel.

This will obviously cause some extra overhead but with the exception of ntfs-3g, most file systems compiled against FUSE tend to be less performance-orientated anyway (eg they might be network bound). But as an aside note, this is also why I don't recommend people using ntfs-3g on low-footprint devices like the Raspberry Pi.

> Oh. This explains quite a bit, because I thought the argument was about whether the OS' usual semantics (group being inherited from directory or from EGID) are like that or not, not where in kernel this behaviour is coded.

> No, I'm making the case that Linux usually works one way, and BSD usually works the other way. I never claimed that anything was coded in any specific kernel subsystem, so don't add the statement yourself.

I know, I was the one that brought code into the debate. I did so very deliberately because software is code and you were making a point about the behavior of software. I'll I'm saying is that I don't agree with you that this behavior is OS-specific because the behavior is defined by the file system. Ie one file system could handle this behavior differently to another file system on the same OS by default. This is why I make a point about "where in the kernel the behavior is coded". It's not just an academic point, it's directly refuting the entire argument you've been having.

Anyhow, since your profile lists you as a developer as well as systems administrator - and you clearly do have a good understanding of the underlying architecture of BSD and Linux - I suggest you play around with a hobby file system yourself. It's good fun and FUSE is dead easy to learn as there will be bindings for most popular languages. I did mine in Go using an API that closely mirrors FUSE's C libraries but I've seen that most higher level languages (eg like Javascript) have FUSE bindings too - if that's your thing instead. Or obviously there's Rust and C/C++ if you prefer to work more directly with FUSE. You don't even need to work with hardware since you could just wrap FUSE around MySQL queries (as one of my latest experiments does) or even contents described in an ASCII file (as my first project did). Because Linux / BSD / whatever doesn't handle the actually writing of the file system (and why would it?), it means it's up to the file system to implement whatever set of behaviors surrounding permissions it wants.

Re:How do groups work on Linux? by dozzie on Wednesday November 22, 2017 @06:31AM (#15756009)
> Also I demonstrated that FreeBSD + ZFS behaves the same as Linux on my system.

No, you didn't. You attempted that, but you failed miserably by not including the most important piece of information: primary group of your shell.

> Perhaps you were using UFS instead of ZFS? Or non-default file system mount options / or other tweaks?

No, it was ZFS, as I stated clearly elsewhere. No options from me whatsoever. Now try the experiment with creating files again, but this time check your process' EGID. It may turn out that your interpretation of your own output was wrong.

> I'll I'm saying is that I don't agree with you that this behavior is OS-specific because the behavior is defined by the file system.

Being defined by a filesystem handling code and being the usual behaviour under particular OS are not the same thing.

Your claim boils down to disagreeing that it's the most prevalent behaviour for the OS, and you justify your position with the behaviour being coded in particular place, no matter that most of the other places for a given OS work in the same way. This is silly stance.

> I suggest you play around with a hobby file system yourself.

And this exercise would give me what, assuming that I haven't implemented a filesystem already?

Re:How do groups work on Linux? by laumars on Wednesday November 22, 2017 @07:10AM (#15756174)
> No, you didn't. You attempted that, but you failed miserably by not including the most important piece of information: primary group of your shell.

I did check that but must have missed it off the post. Sorry about that.

> Being defined by a filesystem handling code and being the usual behaviour under particular OS are not the same thing.

That was my whole bloody point! Glad you finally agree with me. It's the file systems that define that behavior so it's file system-specific not OS-specific (nor Linux-specific) like you previously claimed.

> Your claim boils down to disagreeing that it's the most prevalent behaviour for the OS, and you justify your position with the behaviour being coded in particular place, no matter that most of the other places for a given OS work in the same way. This is silly stance.

No, my point was that it's a file system-specific behavior not an OS-specific behavior. A point you then tried to pass off as your own (above) despite me categorically stating - numerous times throughout the discourse - exactly what my position is. Honestly I think this argument for you is less about facts and more about gradually moving the goal posts so you don't appear wrong.

> And this exercise would give me what

A greater understanding of a topic you've been happy to engage in arguments about yet reluctant to listen to other peoples feedback on. You're clearly an intelligent guy so if you're not going to take the experience of your peers into account (and I cannot blame you for wanting to trust your own experiences) then why not research this topic properly instead?

> assuming that I haven't implemented a filesystem already?

It's a bit late in the discussion to be playing the bluff card. :P

Re:How do groups work on Linux? by dozzie on Wednesday November 22, 2017 @07:48AM (#15756337)
Dude, my whole point revolves around the fact that the most prevalent behaviour for a given OS is OS-specific, which is a fucking tautology. What kind of feedback you expect me to accept from somebody who tries to make me believe the tautology is false?

Not to mention that the somebody tries to imply that I don't have enough knowledge about topic I'm quite sure I have, without ever pointing any error on my side, beside the tautology thing above.