> That sounds pretty overengineered. All you need is a small binary that is setgid shadow, that can take username/password on stdin, and exit 0 if auth succeeds, 1 if auth fails. But we already have that: unix_chkpwd.
That's probably too limited actually. Because you can have a whole bunch of stuff in PAM: LDAP, OTP, Yubikeys, and all kinds of other fancy modules. Doesn't seem that unix_chkpwd handles any of that.
Also, I still think it has to be a network service, because the unix fork model offers lots of avenues for attack. The parent process gets to mess with a whole bunch of things before exec'ing anything, and that list keeps getting longer as the kernel adds features. It's not possible for the child to reliably defend itself against anything the parent might do.
Thus I think for security, a network socket is the best way as it doesn't allow the client process to manipulate the environment of the security service.
Yes, you're right. unix_chkpwd doesn't handle any of that and, in fact, was never intended to handle any of that -- and that's the entire point!
The entire design of PAM is, well, to have separate "modules" that are "pluggable" depending on how you need to handle "authentication" -- LDAP, OTP, Yubikeys, etc.
That is, the pam_unix module (which uses unix_chkpwd) is used when you enter in your (local user account's) password. If you're using something else -- LDAP or NIS or whatever -- for user accounts (i.e., your "passwd" database) there are separate (PAM) modules for that!
> Also, I still think it has to be a network service, ...
No, it really doesn't and, besides, there are other alternatives that would be much better to use instead of a network socket (such as a local UNIX socket, for one).
I really try not to make such remarks here on HN, but in this case it does seem that you have a fundamental misunderstanding of just how this stuff works (which is almost certainly why you're comment has been so downvoted).
xscreensaver-auth uses PAM to authenticate, which (typically) runs a small program called unix_chkpwd, which is setgid shadow (not setuid root), as /etc/shadow is owned by root:shadow and is readable by group.
xscreensaver is one of the very few screen lockers that is reasonably secure, because the bit that locks the screen is pretty minimal and only links with a few fairly-low level X11 libraries, not a full-scale GUI toolkit.
> What we should have instead is some sort of authentication service. A program confined by SELinux to only interact with a socket and the password database, speaking a simple but well designed protocol.
That sounds pretty overengineered. All you need is a small binary that is setgid shadow, that can take username/password on stdin, and exit 0 if auth succeeds, 1 if auth fails. But we already have that: unix_chkpwd.