There is a security issue with using AD/OpenLDAP. That being passwords. An example is you can debug/trace the SSH process and see(log) passwords in clear text.

Now of course this is both a trust and a escalated privilege issue. But a lot harder(impossible) with keys.

There's various methods to put SSH keys in LDAP. LDAP is an extensible database, so you can add a schema to attach SSH keys to a user (various schemas exist) - and the AuthorizedKeysCommand in your sshd_config can be hooked up to any program which can retrieve keys for a given user from anywhere (again, various programs exist).

As an example already put together: https://github.com/jirutka/ssh-ldap-pubkey

If you're using straight passwords, sure, an ssh key (ideally in a directory) is going to beat a password-only login, but that ssh key is still subject to credential theft. I'm in the process of migrating an environment off of ssh-key authentication over to password+otp based kerberos (FreeIPA, though AD makes this easy too). A single password+otp login gets an 8 hour (non-renwable, in this environment) ticket. At the end of those 8 hours, users have to obtain another ticket with a password+otp. What's particularly nifty about this scheme is that it's useful for not only ssh access, but internal https services as well.

You can put keys in LDAP.