I have been using OpenSSH and OpenLDAP to do password authentication for decades and just recently discovered this as an option. It is pretty simple to setup. Basically just extend your ldap schema with the public key attribute and then tell openssh to use it.